yocto初初探

什么是yocto？

• 开源
• 为不同的硬件架构制作基于Linux的系统Image镜像

特性

• Widely Adopted Across the Industry
• Architecture Agnostic（不限架构）
• Images and Code Transfer Easily
• Flexibility（基础版本--->定制版本）
• Ideal for Constrained Embedded and IoT devices
• Comprehensive Toolchain Capabilities
• Mechanism Rules Over Policy
• Uses a Layer Model
• Supports Partial Builds
• Releases According to a Strict Schedule
• Rich Ecosystem of Individuals and Organizations
• Binary Reproducibility
• License Manifest

挑战

• Steep Learning Curve
• Understanding What Changes You Need to Make For Your Design Requires Some Research
• Project Workflow Could Be Confusing
• Working in a Cross-Build Environment Can Feel Unfamiliar
• Initial Build Times Can be Significant

准备

• Git 1.8.3.1 or greater
• tar 1.28 or greater
• Python 3.6.0 or greater.
• gcc 5.0 or greater.

poky编译使用

安装依赖

sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit mesa-common-dev zstd liblz4-tool

上面的pylint3在ubuntu22.04上面没有，我安装了pylint，暂未发现问题

下载poky

git clone git://git.yoctoproject.org/poky

选择一个稳定分支：

git checkout --track -b kirkstone origin/kirkstone
git pull

编译固件

初始化编译环境

$ cd poky
$ source oe-init-build-env

这个操作会生成1个build目录，并在其内部生成conf配置

• build/conf/local.conf
• 该配置文件能够进行各种配置
• 配置内部的注释就是很好的说明文档，可以参考
• 临时修改，不会被git跟踪

source命令支持后面添加build-directory，如下：

source poky/oe-init-build-env [build-directory]

这个非常方便，创建多个build目录，可以并行开发独立的项目，互不干扰

开始编译

bitbake core-image-sato

可以通过下面的命令查看可用的image配置：

fuqiang@fuqiang-virtual:~/workspace/poky$ ls meta*/recipes*/images/*.bb
meta/recipes-core/images/build-appliance-image_15.0.0.bb：虚拟机image，运行在VMware Workstation
meta/recipes-graphics/images/core-image-weston.bb
meta/recipes-core/images/core-image-base.bb
meta/recipes-graphics/images/core-image-weston-sdk.bb
meta/recipes-core/images/core-image-minimal.bb：最小固件，支持启动流程，适合调试和测试kernel和boot laoder
meta/recipes-graphics/images/core-image-x11.bb
meta/recipes-core/images/core-image-minimal-dev.bb：最小固件+header+library，可开发
meta/recipes-rt/images/core-image-rt.bb
meta/recipes-core/images/core-image-minimal-initramfs.bb             
meta/recipes-rt/images/core-image-rt-sdk.bb
meta/recipes-core/images/core-image-minimal-mtdutils.bb              
meta/recipes-sato/images/core-image-sato.bb：支持sato，usrX11，提供一些应用
meta/recipes-core/images/core-image-ptest-all.bb                     
meta/recipes-sato/images/core-image-sato-dev.bb：sato+header+library，应用开发
meta/recipes-core/images/core-image-ptest-fast.bb                    
meta/recipes-sato/images/core-image-sato-sdk.bb：sato+sdk，适合目标机开发
meta/recipes-core/images/core-image-tiny-initramfs.bb                
meta-selftest/recipes-test/images/error-image.bb
meta/recipes-extended/images/core-image-full-cmdline.bb：命令行固件
meta-selftest/recipes-test/images/oe-selftest-image.bb
meta/recipes-extended/images/core-image-kernel-dev.bb                
meta-selftest/recipes-test/images/test-empty-image.bb
meta/recipes-extended/images/core-image-testcontroller.bb            
meta-selftest/recipes-test/images/wic-image-minimal.bb
meta/recipes-extended/images/core-image-testcontroller-initramfs.bb  
meta-skeleton/recipes-multilib/images/core-image-multilib-example.bb

使用qemu仿真

$ runqemu qemux86-64

完整命令如下：

runqemu <machine> <zimage> <filesystems>

yocto相关概念

bitbake

• 解析metadata
• 生成task list
• 执行task list

$ bitbake -h
$ bitbake --help
$ bitbake xxx

$ bitbake <recipe> 构建recipe
$ bitbake <recipe> -c <task> 执行recipe的某个task
$ bitbake <recipe> -c listtasks 列出recipe支持的task

例子：执行matchbox-desktop_1.2.3.bb，如下：

$ bitbake matchbox-desktop

常用的task：

• do_fetch：拉取代码，默认放置在build/conf/local.conf，可配置
• do_unpack：解压代码，放置在build/tmp/work中
• do_patch：源码准备好后，向源码打补丁
• do_configure,do_compile,do_install：install在build/tmp/work/<...>/image
• do_package

bitbake使用的metadata主要分为3类：

• Configuration (the .conf files)
• Classes (the .bbclass files)
• Recipes (the .bb and .bbappend files)

bitbake解析顺序：

• build/conf/bblayers.conf
• meta-xxx/cof/layer.conf
• meta-xxx/conf/bitbake.conf
• build/conf/local.conf

bitbake解决依赖问题：

• DEPENDS:编译时依赖
• RDEPENDS:运行时依赖
• PROVIDES：提供给其他recipe的依赖
• 没有任何依赖的recipe开始并行编译
• 有依赖的recipe开始按照需要的次序串行编译

metadata

查看recipe的环境变量：

bitbake -e <recipe> | grep <variable>

Recipes

xx.bb文件，包含：

• 下载source位置
• patch
• 配置
• 如何编译
• 如何打包

Classes

xx.bbclass文件，包含：

• 抽象通用功能
• 多recipoes间共享

Configurations

xx.conf文件，包含：

• machine configuration options,
• distribution configuration options,
• compiler tuning options,
• general common configuration options,
• user configuration options in conf/local.conf

Layers

查看layer：

bitbake-layers show-layers

输出如下：

uqiang@fuqiang-virtual:~/workspace/poky/build/tmp/work/core2-64-poky-linux/wayland/1.20.0-r0/pkgdata-sysroot/runtime$ bitbake-layers show-layers
NOTE: Starting bitbake server...
layer                 path                                      priority
==========================================================================
meta                  /home/fuqiang/workspace/poky/meta         5
meta-poky             /home/fuqiang/workspace/poky/meta-poky    5
meta-yocto-bsp        /home/fuqiang/workspace/poky/meta-yocto-bsp  5

layer的内部结构如下：

meta-*
├── classes : .bbclass 文件，非必须文件夹
├── conf ： 必须文件夹，配置文件
├── COPYING.MIT : license
├── recipes-* ： 各种recipes

自动生成定制化layer：

$: ./poky/scripts/yocto-layer create newlayer

OpenEmbedded Build System

• User Configuration: metadata you can use to control the build process.
• Metadata Layers: Various layers that provide software, machine, and distro metadata.
• Source Files: Upstream releases, local projects, and SCMs.
• Build System: Processes under the control of BitBake. This block expands on how BitBake fetches source, applies patches, completes compilation, analyzes output for package generation, creates and tests packages, generates images, and generates cross-development tools.
• Package Feeds: Directories containing output packages (RPM, DEB or IPK), which are subsequently used in the construction of an image or Software Development Kit (SDK), produced by the build system. These feeds can also be copied and shared using a web server or other means to facilitate extending or updating existing images on devices at runtime if runtime package management is enabled.
• Images: Images produced by the workflow.
• Application Development SDK: Cross-development tools that are produced along with an image or separately with BitBake.

build目录

编译目录包含：

• conf：配置文件，local.conf，bblayers.conf
• downloads：保存下载的tar包
• sstate-cache：cache用于加速编译
• tmp：临时目录
  • deploy：存放编译输出的内容，包括images，packages，SDKs等
  • sysroots：存放共享库，头文件，utils库等
  • work：解压后的源代码，task的conf，logs等

work目录

• all-poky-linux ： 架构无关的recipe
• core2-64-poky-linux ： 架构相关的recipe
• qemux86_64-poky-linux ： machine相关的recipe
• x86_64-linux ： host sysroot

work目录的组织结构：

<arch>/<recipe name>/<software version>

再看每一个recipe内部的代码结构：

• <sources> ： 源代码
• image ： 安装文件
• packages ： package的内容
• packages-split ： package的内容分开存放
• temp ： task code和执行log

package

支持的package格式：

• RPM ： Red Hat Package Manager，用于Red Hat，Fedora，CentOS等
• DEB ： Debian Package Manager，用于Debian，Ubuntu等
• IPK ： Itsy Package Management System，OpenWrt等
• TAR

指定package的format：

PACKAGE_CLASSES ?= "package_rpm package_deb package_ipk"

package支持脚本操作作为其安装拆卸的一部分：

• preinst : 在package未解压之前执行的操作
• postinst ： 在package安装之后执行的操作
• prerm ： 在package删除之前执行的操作
• postrm ： 在package删除之后执行的操作

shared state cache

• 存储package的snapshot，用于确定是否需要重新编译这个package
• sstate-cache需要定期clean
• 需要完全重新编译，可以删除build/tmp或者同时删除build/tmp和sstate-cache

SDK

• image-based SDK
• generic SDK

image-based SDK

 bitbake core-image-full-cmdline -c populate_sdk

生成如下脚本：

build/tmp/deploy/sdk/pokyeglibc-x86_64-core-image-full-cmdline-armv5te-toolchain-1.6.sh

执行该脚本安装SDK，安装目录如下：

 /opt/poky/1.6

generic SDK

bitbake meta-toolchain

生成如下脚本：

 build/tmp/deploy/sdk/poky-eglibc-x86_64-metatoolchain-armv5te-toolchain-1.6.sh

使用SDK

$: source /opt/poky/1.6/environment-setup-armv5te-poky-linux-gnueabi 
$: ${CC} hello-world.c -o hello-world

或者

$: source /opt/poky/1.6/environment-setup-armv5te-poky-linux-gnueabi 
$: unset LDFLAGS
$: make defconfig
$: make uImage

yocto新手注意事项

• Use Git, not the tarball download：git方便升级
• Get to know the layer index：
• Use existing BSP layers from silicon vendors when possible
• Do not put everything into one layer
• Never modify the POKY layer. Never. Ever. When you update to the next release, you’ll lose all of your work. ALL OF IT.
• Don’t be fooled by documentation searching results
• Understand the basic concepts of how the build system works: the workflow
  • Fetch – get the source code
  • Extract – unpack the sources
  • Patch – apply patches for bug fixes and new capability
  • Configure – set up your environment specifications
  • Build – compile and link
  • Install – copy files to target directories
  • Package – bundle files for installation
• Know that you can generate a dependency graph and learn how to do it
• Here’s how you decode “magic” folder names in tmp/work
• You can build more than just images
• An ambiguous definition: Package vs Recipe
• You will want to learn about and know what’s packaged in the root filesystem.
• Create your own image recipe
• Finally, here is a list of the basic skills you will need as a systems developer. You must be able to:
  • deal with corporate proxies
  • add a package to an image
  • understand the difference between a recipe and package
  • build a package by itself and why that’s useful
  • find out what packages are created by a recipe
  • find out what files are in a package
  • find out what files are in an image
  • add an ssh server to an image (enable transferring of files to target)
  • know the anatomy of a recipe
  • know how to create and use layers
  • find recipes (with the OpenEmbedded Layer index)
  • understand difference between machine and distro settings
  • find and use the right BSP (machine) for your hardware
  • find examples of distro features and know where to set them
  • understanding the task pipeline and executing individual tasks
  • understand devtool and how it simplifies your workflow
  • improve build speeds with shared downloads and shared state cache
  • generate and understand a dependency graph
  • generate and understand bitbake environment
  • build an Extensible SDK for applications development

RbuOS yocto

目录结构

第一二级目录

fuqiang@snode1:~/workspace/th1520$ tree -L 2
.
├── doc    -------------------------------->文档
│   ├── SDK用户用册
│   ├── thead-sdk-doc
│   ├── 器件手册
│   ├── 安全子系统
│   ├── 芯片手册
│   └── 验证板手册
├── image    ------------------------------>现成固件
│   └── light_fm_img
├── tools    ------------------------------>实用工具
│   ├── bin2ext4
│   ├── fastboot
│   ├── fota
│   ├── imagesign
│   └── kdf
└── yocto    ------------------------------>yocto系统目录
    ├── build    -------------------------->编译构建目录
    ├── meta-external-toolchain    -------->工具链layer
    ├── meta-light    --------------------->light系统相关layer
    ├── meta-openembedded    -------------->底层BSP相关layer
    ├── meta-riscv    --------------------->risc-v相关layer
    ├── meta-thead    --------------------->线程相关layer
    ├── openembedded-core    -------------->yocto核心
    └── riscv-toolchain    ---------------->risc-v工具链

24 directories, 0 files

bblayers.conf（yocto/build/light-fm/conf/bblayers.conf）

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "7"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

# YOCTOROOT ="${@os.path.abspath(os.path.join(os.path.join("${TOPDIR}", os.pardir), os.pardir))}"
YOCTOROOT ="${@os.path.abspath("${TOPDIR}/../../")}"

BBLAYERS ?= " \
  ${YOCTOROOT}/openembedded-core/meta \    ---------------->基本配方，包括u-boot，kernel等
  ${YOCTOROOT}/meta-openembedded/meta-oe \
  ${YOCTOROOT}/meta-openembedded/meta-python \
  ${YOCTOROOT}/meta-openembedded/meta-multimedia \
  ${YOCTOROOT}/meta-openembedded/meta-networking \
  ${YOCTOROOT}/meta-external-toolchain \
  ${YOCTOROOT}/meta-riscv \
  ${YOCTOROOT}/meta-thead \
  ${YOCTOROOT}/meta-light \
  ${YOCTOROOT}/meta-openembedded/meta-gnome \
  ${YOCTOROOT}/meta-openembedded/meta-filesystems \
  ${YOCTOROOT}/meta-openembedded/meta-webserver \
  "