yocto 编译流程分析

git clone 一份poky 的工程到本地。

source poky/oe-init-build-env your_build_path

看下 oe-init-build-env 这个shell 脚本都干了些什么:

if [ -z "$ZSH_NAME" ] && [ "x$0" = "x./oe-init-build-env" ]; then
   echo "Error: This script needs to be sourced. Please run as '. ./oe-init-build-env'"
else
   if [ -n "$BASH_SOURCE" ]; then
      OEROOT="`dirname $BASH_SOURCE`"
   elif [ -n "$ZSH_NAME" ]; then
      OEROOT="`dirname $0`"
   else
      OEROOT="`pwd`"
   fi
   OEROOT=`readlink -f "$OEROOT"`
   export OEROOT
   . $OEROOT/scripts/oe-buildenv-internal && \
        $OEROOT/scripts/oe-setup-builddir && \
        [ -n "$BUILDDIR" ] && cd $BUILDDIR
   unset OEROOT
   unset BBPATH
fi

转载: OSChina 上一个博友分析的这段shell 脚本:

http://my.oschina.net/u/158589/blog/70921


第一步,判断该脚本是否是用source或者.的方法运行的。 但是

1["x$0"="x./oe-init-build-env"]
只能对./oe-init-build-env这种执行方式报错,对./yocto/oe-init-build-env是不会报错的。


第二步,设置OEROOT这个变量。当用.或者source去执行脚本时,BASH_SOURCE这个变量会被自动设置到源文件路径。于是dirname $BASH_SOURCE就获得了脚本所在目录。readlink -f $OEROOT获得了绝对路径。

 

第三步,执行oe-buildenv-internal和oe-setup-builddir这两个脚本,并且进入到build目录。

(注意这里的oe-buildenv-internal是用.执行的,而oe-setup-build是fork shell执行的。因为oe-setup-builddir中用到了OEROOT这个变量,所以在此之前,OEROOT必须被export,正如脚本中所做的那样。)

 

第四步,unset一些变量。因为.或者source的执行方式是在原shell中执行,并不fork shell,所以如果不unset,会一直留在该shell中。

 

chenqi@chenqi-laptop ~/MyPro/ShellScript/yocto $ . ./oe-init-build-env
BASH_SOURCE = ./oe-init-build-env, OEROOT = .
OEROOT = /home/chenqi/MyPro/ShellScript/yocto

 

chenqi@chenqi-laptop ~/MyPro/ShellScript $ source yocto/oe-init-build-env
BASH_SOURCE = yocto/oe-init-build-env, OEROOT = yocto
OEROOT = /home/chenqi/MyPro/ShellScript/yocto

 

可见,无论在哪个目录下执行,最后获得的OEROOT的绝对路径都是一致的。(主要利用BASH_SOURCE, dirname, readlink)。


. $OEROOT/scripts/oe-buildenv-internal

oe-buildenv-internal 这个shell 脚本主要是设置环境变量(poky/bitbake/bin/ 和 poky/scripts 这两个路径加入到PATH中。bitbake 命令就在poky/bitbake/bin/路径下面。),如下:

PATH="${OEROOT}/scripts:$BITBAKEDIR/bin/:$PATH"
unset BITBAKEDIR

# Used by the runqemu script
export BUILDDIR
export PATH
export BB_ENV_EXTRAWHITE="MACHINE DISTRO TCMODE TCLIBC HTTP_PROXY http_proxy \
HTTPS_PROXY https_proxy FTP_PROXY ftp_proxy FTPS_PROXY ftps_proxy ALL_PROXY \
all_proxy NO_PROXY no_proxy SSH_AGENT_PID SSH_AUTH_SOCK BB_SRCREV_POLICY \
SDKMACHINE BB_NUMBER_THREADS BB_NO_NETWORK PARALLEL_MAKE GIT_PROXY_COMMAND \
SOCKS5_PASSWD SOCKS5_USER SCREENDIR STAMPS_DIR"

$OEROOT/scripts/oe-setup-builddir

oe-setup-builddir 这个shell 脚本,创建编译目录,判断当前编译目录下面是否存在conf/local.conf 文件,如果不存在local.conf 配置文件的话,通过Template模板,sample 生成一个local.conf。

下面这段shell 脚本的意思是首先检查 meta-yocto 目录下面是否存在conf/ 如果存在的话就用meta-yocto/conf/  下面的 local.conf.sample 和 bblayers.conf, 如果不存在的话就到meta/conf 下面去找 local.conf.sample 和 bblayer.conf。


TEMPLATECONF=${TEMPLATECONF:-meta-yocto/conf}

# 
# $TEMPLATECONF can point to a directory for the template local.conf & bblayers.conf
#
if [ "x" != "x$TEMPLATECONF" ]; then
    if ! (test -d "$TEMPLATECONF"); then
        # Allow TEMPLATECONF=meta-xyz/conf as a shortcut
        if [ -d "$OEROOT/$TEMPLATECONF" ]; then
            TEMPLATECONF="$OEROOT/$TEMPLATECONF"
        fi
        if ! (test -d "$TEMPLATECONF"); then
            echo >&2 "Error: '$TEMPLATECONF' must be a directory containing local.conf & bblayers.conf"
            return
        fi
    fi
    OECORELAYERCONF="$TEMPLATECONF/bblayers.conf.sample"
    OECORELOCALCONF="$TEMPLATECONF/local.conf.sample"
    OECORENOTESCONF="$TEMPLATECONF/conf-notes.txt"
fi

if [ "x" = "x$OECORELOCALCONF" ]; then
    OECORELOCALCONF="$OEROOT/meta/conf/local.conf.sample"
fi
if ! (test -r "$BUILDDIR/conf/local.conf"); then
cat <<EOM
You had no conf/local.conf file. This configuration file has therefore been
created for you with some default values. You may wish to edit it to use a 
different MACHINE (target hardware) or enable parallel build options to take 
advantage of multiple cores for example. See the file for more information as 
common configuration options are commented.

The Yocto Project has extensive documentation about OE including a reference manual
which can be found at:
    http://yoctoproject.org/documentation

For more information about OpenEmbedded see their website:
    http://www.openembedded.org/

EOM
    cp -f $OECORELOCALCONF $BUILDDIR/conf/local.conf
fi
if [ "x" = "x$OECORELAYERCONF" ]; then
    OECORELAYERCONF="$OEROOT/meta/conf/bblayers.conf.sample"
fi
if ! (test -r "$BUILDDIR/conf/bblayers.conf"); then
cat <<EOM
You had no conf/bblayers.conf file. The configuration file has been created for
you with some default values. To add additional metadata layers into your
configuration please add entries to this file.

The Yocto Project has extensive documentation about OE including a reference manual
which can be found at:
    http://yoctoproject.org/documentation

For more information about OpenEmbedded see their website:
    http://www.openembedded.org/


EOM

    # Put the abosolute path to the layers in bblayers.conf so we can run
    # bitbake without the init script after the first run
    sed "s|##COREBASE##|$OEROOT|g" $OECORELAYERCONF > $BUILDDIR/conf/bblayers.conf
fi

至此, build_path 下面的conf/local.conf 以及 bblayer.conf 文件就 创建好了,接下来就可以执行 bitbake target 编译了。



----------------------------------------------------------------  华丽丽的分割线 ---------------------------------------------------------


现在开始分析bitbake 编译流程:

bitbake --help 可以看到bitbake 后面可以跟的选项参数:

yocto_build$ bitbake --help
Usage: bitbake [options] [package ...]

Executes the specified task (default is 'build') for a given set of BitBake files.
It expects that BBFILES is defined, which is a space separated list of files to
be executed.  BBFILES does support wildcards.
Default BBFILES are the .bb files in the current directory.

Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  -b BUILDFILE, --buildfile=BUILDFILE
                        execute the task against this .bb file, rather than a
                        package from BBFILES. Does not handle any
                        dependencies.
  -k, --continue        continue as much as possible after an error. While the
                        target that failed, and those that depend on it,
                        cannot be remade, the other dependencies of these
                        targets can be processed all the same.
  -a, --tryaltconfigs   continue with builds by trying to use alternative
                        providers where possible.
  -f, --force           force run of specified cmd, regardless of stamp status
  -c CMD, --cmd=CMD     Specify task to execute. Note that this only executes
                        the specified task for the providee and the packages
                        it depends on, i.e. 'compile' does not implicitly call
                        stage for the dependencies (IOW: use only if you know
                        what you are doing). Depending on the base.bbclass a
                        listtasks tasks is defined and will show available
                        tasks
  -C INVALIDATE_STAMP, --clear-stamp=INVALIDATE_STAMP
                        Invalidate the stamp for the specified cmd such as
                        'compile' and run the default task for the specified
                        target(s)
  -r PREFILE, --read=PREFILE
                        read the specified file before bitbake.conf
  -R POSTFILE, --postread=POSTFILE
                        read the specified file after bitbake.conf
  -v, --verbose         output more chit-chat to the terminal
  -D, --debug           Increase the debug level. You can specify this more
                        than once.
  -n, --dry-run         don't execute, just go through the motions
  -S, --dump-signatures
                        don't execute, just dump out the signature
                        construction information
  -p, --parse-only      quit after parsing the BB files (developers only)
  -s, --show-versions   show current and preferred versions of all recipes
  -e, --environment     show the global or per-package environment (this is
                        what used to be bbread)
  -g, --graphviz        emit the dependency trees of the specified packages in
                        the dot syntax, and the pn-buildlist to show the build
                        list
  -I EXTRA_ASSUME_PROVIDED, --ignore-deps=EXTRA_ASSUME_PROVIDED
                        Assume these dependencies don't exist and are already
                        provided (equivalent to ASSUME_PROVIDED). Useful to
                        make dependency graphs more appealing
  -l DEBUG_DOMAINS, --log-domains=DEBUG_DOMAINS
                        Show debug logging for the specified logging domains
  -P, --profile         profile the command and print a report
  -u UI, --ui=UI        userinterface to use
  -t SERVERTYPE, --servertype=SERVERTYPE
                        Choose which server to use, none, process or xmlrpc
  --revisions-changed   Set the exit code depending on whether upstream
                        floating revisions have changed or not
  --server-only         Run bitbake without UI,  the frontend can connect with
                        bitbake server itself
  -B BIND, --bind=BIND  The name/address for the bitbake server to bind to
  --no-setscene         Do not run any setscene tasks, forces builds

那么,bitbake 对于这些个选项参数是怎么处理的呢。 bitbake target -xxx -xxxx -xx这样其实就是执行poky/bitbake/bitbake 这个python 脚本:

这个python 脚本相对简单,可以看下最后的这段代码:

通过下面这个 if 判断,调用 main 函数执行,使用 python 库中的

optparse.OptionParser
对选项参数进行处理。


if __name__ == "__main__":
    try:
        ret = main()

#!/usr/bin/env python
# ex:ts=4:sw=4:sts=4:et
# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-

import os
import sys, logging
sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(__file__)),
                                'lib'))

import optparse
import warnings
from traceback import format_exception
try:
    import bb
except RuntimeError as exc:
    sys.exit(str(exc))
from bb import event
import bb.msg
from bb import cooker
from bb import ui
from bb import server

__version__ = "1.17.1"
logger = logging.getLogger("BitBake")

# Unbuffer stdout to avoid log truncation in the event
# of an unorderly exit as well as to provide timely
# updates to log files for use with tail
try:
    if sys.stdout.name == '<stdout>':
        sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0)
except:
    pass

class BBConfiguration(object):
    """
    Manages build options and configurations for one run
    """

    def __init__(self, options):
        for key, val in options.__dict__.items():
            setattr(self, key, val)
        self.pkgs_to_build = []

*** ***

def main():
    parser = optparse.OptionParser(
        version = "BitBake Build Tool Core version %s, %%prog version %s" % (bb.__version__, __version__),
        usage = """%prog [options] [package ...]

Executes the specified task (default is 'build') for a given set of BitBake files.
It expects that BBFILES is defined, which is a space separated list of files to
be executed.  BBFILES does support wildcards.
Default BBFILES are the .bb files in the current directory.""")

    parser.add_option("-b", "--buildfile", help = "execute the task against this .bb file, rather than a package from BBFILES. Does not handle any dependencies.",
               action = "store", dest = "buildfile", default = None)

    parser.add_option("-k", "--continue", help = "continue as much as possible after an error. While the target that failed, and those that depend on it, cannot be remade, the other dependencies of these targets can be processed all the same.",
               action = "store_false", dest = "abort", default = True)

    parser.add_option("-a", "--tryaltconfigs", help = "continue with builds by trying to use alternative providers where possible.",
               action = "store_true", dest = "tryaltconfigs", default = False)

    parser.add_option("-f", "--force", help = "force run of specified cmd, regardless of stamp status",
               action = "store_true", dest = "force", default = False)

    parser.add_option("-c", "--cmd", help = "Specify task to execute. Note that this only executes the specified task for the providee and the packages it depends on, i.e. 'compile' does not implicitly call stage for the dependencies (IOW: use only if you know what you are doing). Depending on the base.bbclass a listtasks tasks is defined and will show available tasks",
               action = "store", dest = "cmd")

    parser.add_option("-C", "--clear-stamp", help = "Invalidate the stamp for the specified cmd such as 'compile' and run the default task for the specified target(s)",
                action = "store", dest = "invalidate_stamp")

    parser.add_option("-r", "--read", help = "read the specified file before bitbake.conf",
               action = "append", dest = "prefile", default = [])

    parser.add_option("-R", "--postread", help = "read the specified file after bitbake.conf",
                      action = "append", dest = "postfile", default = [])

    parser.add_option("-v", "--verbose", help = "output more chit-chat to the terminal",
               action = "store_true", dest = "verbose", default = False)

    parser.add_option("-D", "--debug", help = "Increase the debug level. You can specify this more than once.",
               action = "count", dest="debug", default = 0)

    parser.add_option("-n", "--dry-run", help = "don't execute, just go through the motions",
               action = "store_true", dest = "dry_run", default = False)

    parser.add_option("-S", "--dump-signatures", help = "don't execute, just dump out the signature construction information",
               action = "store_true", dest = "dump_signatures", default = False)

    parser.add_option("-p", "--parse-only", help = "quit after parsing the BB files (developers only)",
               action = "store_true", dest = "parse_only", default = False)

    parser.add_option("-s", "--show-versions", help = "show current and preferred versions of all recipes",
               action = "store_true", dest = "show_versions", default = False)

    parser.add_option("-e", "--environment", help = "show the global or per-package environment (this is what used to be bbread)",
               action = "store_true", dest = "show_environment", default = False)

    parser.add_option("-g", "--graphviz", help = "emit the dependency trees of the specified packages in the dot syntax, and the pn-buildlist to show the build list",
                action = "store_true", dest = "dot_graph", default = False)

    parser.add_option("-I", "--ignore-deps", help = """Assume these dependencies don't exist and are already provided (equivalent to ASSUME_PROVIDED). Useful to make dependency graphs more appealing""",
                action = "append", dest = "extra_assume_provided", default = [])

    parser.add_option("-l", "--log-domains", help = """Show debug logging for the specified logging domains""",
                action = "append", dest = "debug_domains", default = [])

    parser.add_option("-P", "--profile", help = "profile the command and print a report",
               action = "store_true", dest = "profile", default = False)

    parser.add_option("-u", "--ui", help = "userinterface to use",
               action = "store", dest = "ui")

    parser.add_option("-t", "--servertype", help = "Choose which server to use, none, process or xmlrpc",
               action = "store", dest = "servertype")

    parser.add_option("", "--revisions-changed", help = "Set the exit code depending on whether upstream floating revisions have changed or not",
               action = "store_true", dest = "revisions_changed", default = False)

    parser.add_option("", "--server-only", help = "Run bitbake without UI,  the frontend can connect with bitbake server itself",
               action = "store_true", dest = "server_only", default = False)

    parser.add_option("-B", "--bind", help = "The name/address for the bitbake server to bind to",
               action = "store", dest = "bind", default = False)
    parser.add_option("", "--no-setscene", help = "Do not run any setscene tasks, forces builds",
               action = "store_true", dest = "nosetscene", default = False)
    options, args = parser.parse_args(sys.argv)
    
    print "+++++ qc test +++++ options:", options
    print "+++++ qc test +++++ args:", args

    configuration = BBConfiguration(options)
    configuration.pkgs_to_build.extend(args[1:])

    print "+++++ qc test +++++ pkgs_to_build:",configuration.pkgs_to_build
    
    *** ***
   
    # Ensure logging messages get sent to the UI as events
    handler = bb.event.LogHandler()
    logger.addHandler(handler)

    # Before we start modifying the environment we should take a pristine
    # copy for possible later use
    initialenv = os.environ.copy()
    # Clear away any spurious environment variables while we stoke up the cooker
    cleanedvars = bb.utils.clean_environment()

    server = server.BitBakeServer()
    if configuration.bind:
        server.initServer((configuration.bind, 0))
    else:
        server.initServer()

    idle = server.getServerIdleCB()

    cooker = bb.cooker.BBCooker(configuration, idle, initialenv)
    cooker.parseCommandLine()

    server.addcooker(cooker)
    server.saveConnectionDetails()
    server.detach()

    # Should no longer need to ever reference cooker
    del cooker

    logger.removeHandler(handler)

    if not configuration.server_only:
        # Setup a connection to the server (cooker)
        server_connection = server.establishConnection()

        # Restore the environment in case the UI needs it
        for k in cleanedvars:
            os.environ[k] = cleanedvars[k]

        try:
            return server.launchUI(ui_main, server_connection.connection, server_connection.events)
        finally:
            bb.event.ui_queue = []
            server_connection.terminate()
    else:
        print("server address: %s, server port: %s" % (server.serverinfo.host, server.serverinfo.port))

    return 1

if __name__ == "__main__":
    try:
        ret = main()
    except Exception:
        ret = 1
        import traceback
        traceback.print_exc(5)
    sys.exit(ret)


 这里调用到了 parseConfigurationFiles() 函数,这个函数用于解析 prefiles, postfiles (这两个变量都是空,需要通过执行bitbake -r -R 指定才行),解析 layer.conf, bblayer.conf, bitbake.conf, 以及 bb 文件中 inherit 的 bbclass,以及包含base.bbclass。

   def parseConfigurationFiles(self, prefiles, postfiles):
        data = self.configuration.data
        bb.parse.init_parser(data)

        # Parse files for loading *before* bitbake.conf and any includes
        for f in prefiles:
            data = _parse(f, data)

        layerconf = self._findLayerConf()
        if layerconf:
            parselog.debug(2, "Found bblayers.conf (%s)", layerconf)
            data = _parse(layerconf, data)

            layers = (data.getVar('BBLAYERS', True) or "").split()

            data = bb.data.createCopy(data)
            for layer in layers:
                parselog.debug(2, "Adding layer %s", layer)
                data.setVar('LAYERDIR', layer)
                data = _parse(os.path.join(layer, "conf", "layer.conf"), data)
                data.expandVarref('LAYERDIR')

            data.delVar('LAYERDIR')

        if not data.getVar("BBPATH", True):
            raise SystemExit("The BBPATH variable is not set")

        data = _parse(os.path.join("conf", "bitbake.conf"), data)

        # Parse files for loading *after* bitbake.conf and any includes
        for p in postfiles:
            data = _parse(p, data)

        # Handle any INHERITs and inherit the base class
        bbclasses  = ["base"] + (data.getVar('INHERIT', True) or "").split()
        for bbclass in bbclasses:
            data = _inherit(bbclass, data)

        # Nomally we only register event handlers at the end of parsing .bb files
        # We register any handlers we've found so far here...
        for var in data.getVar('__BBHANDLERS') or []:
            bb.event.register(var, data.getVar(var))

        if data.getVar("BB_WORKERCONTEXT", False) is None:
            bb.fetch.fetcher_init(data)
        bb.codeparser.parser_cache_init(data)
        bb.event.fire(bb.event.ConfigParsed(), data)

        if data.getVar("BB_INVALIDCONF") is True:
            data.setVar("BB_INVALIDCONF", False)
            self.parseConfigurationFiles(self.configuration.prefile,
                                         self.configuration.postfile)
        else:
            bb.parse.init_parser(data)
            data.setVar('BBINCLUDED',bb.parse.get_file_depends(data))
            self.configuration.data = data
            self.configuration.data_hash = data.get_hash()


将所有的conf, bb, bbclass, etc 文件解析完毕之后,处理一下这些数据,提取出task list ,之后就是执行这些task 了。

下面以linux-yocto 为例看下如何找到这些task 的先后关系:

yocto_build/tmp/work/qemuppc-poky-linux/linux-yocto/3.4.36+gitAUTOINC+80b4b5110dca5184b57e85f1e775fb006a2e85ad_ddbc382cbc45a009e9ce17f7d448fcfd050ab5fc-r4.3/temp/log.task_order

这个文件显示了编译 linux-yocto 执行的task 以及 task 的执行顺序。

do_fetch (24333): log.do_fetch.24333
do_unpack (24359): log.do_unpack.24359
do_kernel_checkout (24371): log.do_kernel_checkout.24371
do_validate_branches (24399): log.do_validate_branches.24399
do_patch (24438): log.do_patch.24438
do_populate_lic (7751): log.do_populate_lic.7751
do_kernel_configme (7750): log.do_kernel_configme.7750
do_configure (18091): log.do_configure.18091
do_kernel_configcheck (18191): log.do_kernel_configcheck.18191
do_compile (23327): log.do_compile.23327
do_compile_kernelmodules (11394): log.do_compile_kernelmodules.11394
do_uboot_mkimage (11396): log.do_uboot_mkimage.11396
do_kernel_link_vmlinux (11397): log.do_kernel_link_vmlinux.11397
do_sizecheck (11395): log.do_sizecheck.11395
do_install (24128): log.do_install.24128
do_package (13631): log.do_package.13631
do_deploy (13632): log.do_deploy.13632
do_populate_sysroot (13633): log.do_populate_sysroot.13633
do_packagedata (16431): log.do_packagedata.16431
do_package_write_rpm (16452): log.do_package_write_rpm.16452
do_listtasks (7391): log.do_listtasks.7391

接下来介绍如何验证这些task 以及顺序:

poky/meta/recipes-kernel/linux/linux-yocto_3.4.bb

require recipes-kernel/linux/linux-yocto.inc

KBRANCH_DEFAULT = "standard/base"
KBRANCH = "${KBRANCH_DEFAULT}"

SRCREV_machine_qemuarm ?= "7cc80532306889b75619f8a1b713048e25f59e19"
SRCREV_machine_qemumips  ?= "debce6677988e03b50c369aba5861d4f9b2e557d"
SRCREV_machine_qemuppc ?= "ddbc382cbc45a009e9ce17f7d448fcfd050ab5fc"
SRCREV_machine_qemux86 ?= "c994390cfa28339cbc1ec3b56eeec83a5fa75bb7"
SRCREV_machine_qemux86-64 ?= "c994390cfa28339cbc1ec3b56eeec83a5fa75bb7"
SRCREV_machine ?= "c994390cfa28339cbc1ec3b56eeec83a5fa75bb7"
SRCREV_meta ?= "80b4b5110dca5184b57e85f1e775fb006a2e85ad"

SRC_URI = "git://git.yoctoproject.org/linux-yocto-3.4.git;protocol=git;bareclone=1;branch=${KBRANCH},${KMETA};name=machine,meta"

LINUX_VERSION ?= "3.4.36"

PR = "${INC_PR}.3"
PV = "${LINUX_VERSION}+git${SRCPV}"

KMETA = "meta"

COMPATIBLE_MACHINE = "qemuarm|qemux86|qemuppc|qemumips|qemux86-64"

# Functionality flags
KERNEL_FEATURES_append = " features/netfilter/netfilter.scc"
KERNEL_FEATURES_append_qemux86=" cfg/sound.scc"
KERNEL_FEATURES_append_qemux86-64=" cfg/sound.scc"
KERNEL_FEATURES_append_qemux86=" cfg/paravirt_kvm.scc"
KERNEL_FEATURES_append = " ${@bb.utils.contains("TUNE_FEATURES", "mx32", " cfg/x32.scc", "" ,d)}"


poky/meta/recipes-kernel/linux/linux-yocto.inc


DESCRIPTION = "Yocto Kernel"
SECTION = "kernel"
LICENSE = "GPLv2"

LIC_FILES_CHKSUM = "file://COPYING;md5=d7810fab7487fb0aad327b76f1be7cd7"

INC_PR = "r4"

# A KMACHINE is the mapping of a yocto $MACHINE to what is built
# by the kernel. This is typically the branch that should be built,
# and it can be specific to the machine or shared
# KMACHINE = "UNDEFINED"

LINUX_KERNEL_TYPE ?= "standard"

# KMETA ?= ""
KBRANCH ?= "master"
KMACHINE ?= "${MACHINE}"
SRCREV_FORMAT ?= "meta_machine"

LINUX_VERSION_EXTENSION ?= "-yocto-${LINUX_KERNEL_TYPE}"

do_patch[depends] = "kern-tools-native:do_populate_sysroot"

addtask kernel_configme before do_configure after do_patch

# Pick up shared functions
inherit kernel
inherit kernel-yocto
require linux-dtb.inc


B = "${WORKDIR}/linux-${MACHINE}-${LINUX_KERNEL_TYPE}-build"

do_install_append(){
        if [ -n "${KMETA}" ]; then
                rm -rf ${STAGING_KERNEL_DIR}/${KMETA}
        fi
}

# extra tasks
addtask kernel_link_vmlinux after do_compile before do_install
addtask validate_branches before do_patch after do_kernel_checkout
addtask kernel_configcheck after do_configure before do_compile


poky/meta/classes/base.bbclass

poky/meta/classes/kernel.bbclass

poky/meta/classes/kernel-yocto.bbclass

这是 linux-yocto 所有相关的bb, bbclass 文件,所有的task 都是通过addtasks 关键字添加的,它们之间先后关系构成了所谓的依赖关系,第一个task,在这几个bb, bbclass 文件里面肯定没有 addtask before the_1st_task 这样的语句。

poky/meta/classes$ grep -nr "addtask . | grep "patch"

就按照上面这样的方法不断的grep 就能验证出log.task_order 里面显示的task,以及 正确的执行顺序。

ex. $ bitbake linux-yocto -c cleanall 执行 cleanall task 需要运行的tasklist:

poky/meta/classes$ grep -nr "addtask" . | grep "clean"
./base.bbclass:636:addtask cleansstate after do_clean
./base.bbclass:637:addtask qc_test after do_cleansstate
./base.bbclass:648:addtask cleanall after do_cleansstate
./utility-tasks.bbclass:16:addtask clean

其中,qc_test task 是我自己加的测试 task:

addtask cleansstate after do_clean
addtask qc_test after do_cleansstate

do_qc_test() {
        echo "qc hello base.bbclass !"
        echo "qc   test !!!!!~~~~ "
}

这样,构建起来的tasklist 如下:

cleanclean
cleansstatecleansstate
cleanallqc_test

如此,可以看出,执行 cleanall task 的话,clean, cleansstate task 都要执行,因为具有依赖关系,相反,qc_test 和 cleanall 虽然都是 after do_cleansstate ,但是二者之间没有依赖关系。其它的task 的 runqueue list 也是这样得到。


bitbake 如果不刻意指定要执行的task 的话,默认执行的是build 操作,而这个操作是针对一系列的bb 文件,这些文件是在BBFILES定义的。看一下BBFILES 这个变量的来历,方法如下

/poky/bitbake$ grep -nr "BBFILES" .

直接在bitbake 目录下面搜索有谁对BBFILES 这个变量进行了赋值操作,这样就能定位到./lib/bb/cooker.py

反向推理一下:

搜集bbfiles 函数:

    def collect_bbfiles( self ):
        """Collect all available .bb build files"""
        parsed, cached, skipped, masked = 0, 0, 0, 0

        collectlog.debug(1, "collecting .bb files")

        files = (data.getVar( "BBFILES", self.configuration.data, True) or "").split()
        data.setVar("BBFILES", " ".join(files), self.configuration.data)

        # Sort files by priority
        files.sort( key=lambda fileitem: self.calc_bbfile_priority(fileitem) )

    def matchFile(self, buildfile):
        """
        Find the .bb file which matches the expression in 'buildfile'.
        Raise an error if multiple files
        """
        matches = self.matchFiles(buildfile)


   def buildFile(self, buildfile, task):
        """
        Build the file matching regexp buildfile
        """

        # Too many people use -b because they think it's how you normally
        # specify a target to be built, so show a warning
        bb.warn("Buildfile specified, dependencies will not be handled. If this is not what you want, do not use -b / --buildfile.")

        # Parse the configuration here. We need to do it explicitly here since
        # buildFile() doesn't use the cache
        self.parseConfiguration()

        # If we are told to do the None task then query the default task
        if (task == None):
            task = self.configuration.cmd

        fn, cls = bb.cache.Cache.virtualfn2realfn(buildfile)
        fn = self.matchFile(fn)

        self.buildSetVars()


   def parseCommandLine(self):
        # Parse any commandline into actions
        self.commandlineAction = {'action':None, 'msg':None}
        if self.configuration.show_environment:
            if 'world' in self.configuration.pkgs_to_build:
                self.commandlineAction['msg'] = "'world' is not a valid target for --environment."
            elif 'universe' in self.configuration.pkgs_to_build:
                self.commandlineAction['msg'] = "'universe' is not a valid target for --environment."
            elif len(self.configuration.pkgs_to_build) > 1:
                self.commandlineAction['msg'] = "Only one target can be used with the --environment option."
            elif self.configuration.buildfile and len(self.configuration.pkgs_to_build) > 0:
                self.commandlineAction['msg'] = "No target should be used with the --environment and --buildfile options."
            elif len(self.configuration.pkgs_to_build) > 0:
                self.commandlineAction['action'] = ["showEnvironmentTarget", self.configuration.pkgs_to_build]
                self.configuration.data.setVar("BB_CONSOLELOG", None)
            else:
                self.commandlineAction['action'] = ["showEnvironment", self.configuration.buildfile]
                self.configuration.data.setVar("BB_CONSOLELOG", None)
        elif self.configuration.buildfile is not None:
            self.commandlineAction['action'] = ["buildFile", self.configuration.buildfile, self.configuration.cmd]
        elif self.configuration.revisions_changed:
            self.commandlineAction['action'] = ["compareRevisions"]
        elif self.configuration.show_versions:
            self.commandlineAction['action'] = ["showVersions"]
        elif self.configuration.parse_only:
            self.commandlineAction['action'] = ["parseFiles"]
        elif self.configuration.dot_graph:
            if self.configuration.pkgs_to_build:
                self.commandlineAction['action'] = ["generateDotGraph", self.configuration.pkgs_to_build, self.configuration.cmd]
            else:
                self.commandlineAction['msg'] = "Please specify a package name for dependency graph generation."
        else:
            if self.configuration.pkgs_to_build:
                self.commandlineAction['action'] = ["buildTargets", self.configuration.pkgs_to_build, self.configuration.cmd]
            else:
                #self.commandlineAction['msg'] = "Nothing to do.  Use 'bitbake world' to build everything, or run 'bitbake --help' for usage information."
                self.commandlineAction = None

仔细看下上面这个函数,是不是和最初在bitbake 这个python 根脚本中定义的main 函数有很多相似之处,这样就能猜到 bitbake -b xxx 这个指令回执行到buildFile 这个python 函数。同样可以知道 bitbake -e 可以看到所有的环境变量,包括BBFILES 变量的值,因为它执行了 showEnvrioment 这个python 函数。

强烈建议把 bitbake -e > ~/bitbake_-e.txt  重定向到文件中好好看看。

 

def main():
    parser = optparse.OptionParser(
        version = "BitBake Build Tool Core version %s, %%prog version %s" % (bb.__version__, __version__),
        usage = """%prog [options] [package ...]

Executes the specified task (default is 'build') for a given set of BitBake files.
It expects that BBFILES is defined, which is a space separated list of files to
be executed.  BBFILES does support wildcards.
Default BBFILES are the .bb files in the current directory.""")

    parser.add_option("-b", "--buildfile", help = "execute the task against this .bb file, rather than a package from BBFILES. Does not handle any dependencies.",
               action = "store", dest = "buildfile", default = None)

    parser.add_option("-k", "--continue", help = "continue as much as possible after an error. While the target that failed, and those that depend on it, cannot be remade, the other dependencies of these targets can be processed all the same.",
               action = "store_false", dest = "abort", default = True)

    parser.add_option("-a", "--tryaltconfigs", help = "continue with builds by trying to use alternative providers where possible.",
               action = "store_true", dest = "tryaltconfigs", default = False)

    parser.add_option("-f", "--force", help = "force run of specified cmd, regardless of stamp status",
               action = "store_true", dest = "force", default = False)

    parser.add_option("-c", "--cmd", help = "Specify task to execute. Note that this only executes the specified task for the providee and the packages it depends on, i.e. 'compile' does not implicitly call stage for the dependencies (IOW: use only if you know what you are doing). Depending on the base.bbclass a listtasks tasks is defined and will show available tasks",
               action = "store", dest = "cmd")

OK.

再看一下 parseCommandLine 这个函数if elif elif ,,, else 通过不断的探测bitbake 的选项参数,如果没有选项参数,只有target 走到buildTargets 这个else 里面,接下来要执行的函数是buildTargets.

   def parseCommandLine(self):
        # Parse any commandline into actions
        self.commandlineAction = {'action':None, 'msg':None}
        if self.configuration.show_environment:
            if 'world' in self.configuration.pkgs_to_build:
                self.commandlineAction['msg'] = "'world' is not a valid target for --environment."
            elif 'universe' in self.configuration.pkgs_to_build:
                self.commandlineAction['msg'] = "'universe' is not a valid target for --environment."
            elif len(self.configuration.pkgs_to_build) > 1:
                self.commandlineAction['msg'] = "Only one target can be used with the --environment option."
            elif self.configuration.buildfile and len(self.configuration.pkgs_to_build) > 0:
                self.commandlineAction['msg'] = "No target should be used with the --environment and --buildfile options."
            elif len(self.configuration.pkgs_to_build) > 0:
                self.commandlineAction['action'] = ["showEnvironmentTarget", self.configuration.pkgs_to_build]
                self.configuration.data.setVar("BB_CONSOLELOG", None)
            else:
                self.commandlineAction['action'] = ["showEnvironment", self.configuration.buildfile]
                self.configuration.data.setVar("BB_CONSOLELOG", None)
        elif self.configuration.buildfile is not None:
            self.commandlineAction['action'] = ["buildFile", self.configuration.buildfile, self.configuration.cmd]
        elif self.configuration.revisions_changed:
            self.commandlineAction['action'] = ["compareRevisions"]
        elif self.configuration.show_versions:
            self.commandlineAction['action'] = ["showVersions"]
        elif self.configuration.parse_only:
            self.commandlineAction['action'] = ["parseFiles"]
        elif self.configuration.dot_graph:
            if self.configuration.pkgs_to_build:
                self.commandlineAction['action'] = ["generateDotGraph", self.configuration.pkgs_to_build, self.configuration.cmd]
            else:
                self.commandlineAction['msg'] = "Please specify a package name for dependency graph generation."
        else:
            if self.configuration.pkgs_to_build:
                self.commandlineAction['action'] = ["buildTargets", self.configuration.pkgs_to_build, self.configuration.cmd]
            else:
                #self.commandlineAction['msg'] = "Nothing to do.  Use 'bitbake world' to build everything, or run 'bitbake --help' for usage information."
                self.commandlineAction = None

看一个特定的package 编译流程,拿kernel 看吧:

从bitbake_-e.txt 环境中搜到:

PREFERRED_PROVIDER_virtual/kernel="linux-yocto"

PREFERRED_VERSION_linux-yocto="3.4%"

这样就可以在poky 目录下面搜索 名字为 linux-yocto*.bb* 的 bb和bbappend, 搜出来以后再取 3.4版本的那些bb 和 bbappend。

/poky$ find -name "linux-yocto*.bb*"

./meta/recipes-kernel/linux/linux-yocto_3.8.bb
./meta/recipes-kernel/linux/linux-yocto-dev.bb
./meta/recipes-kernel/linux/linux-yocto-rt_3.4.bb
./meta/recipes-kernel/linux/linux-yocto-rt_3.8.bb
./meta/recipes-kernel/linux/linux-yocto-rt_3.2.bb
./meta/recipes-kernel/linux/linux-yocto-tiny_3.2.bb
./meta/recipes-kernel/linux/linux-yocto_3.4.bb
./meta/recipes-kernel/linux/linux-yocto_3.2.bb
./meta/recipes-kernel/linux/linux-yocto-tiny_3.4.bb
./meta/recipes-kernel/linux/linux-yocto-tiny_3.8.bb
./meta-yocto-bsp/recipes-kernel/linux/linux-yocto_3.8.bbappend
./meta-yocto-bsp/recipes-kernel/linux/linux-yocto_3.2.bbappend
./meta-yocto-bsp/recipes-kernel/linux/linux-yocto_3.4.bbappend
./meta-skeleton/recipes-kernel/linux/linux-yocto-custom.bb

查看./meta/recipes-kernel/linux/linux-yocto_3.4.bb

require recipes-kernel/linux/linux-yocto.inc

KBRANCH_DEFAULT = "standard/base"
KBRANCH = "${KBRANCH_DEFAULT}"

SRCREV_machine_qemuarm ?= "7cc80532306889b75619f8a1b713048e25f59e19"
SRCREV_machine_qemumips  ?= "debce6677988e03b50c369aba5861d4f9b2e557d"
SRCREV_machine_qemuppc ?= "ddbc382cbc45a009e9ce17f7d448fcfd050ab5fc"
SRCREV_machine_qemux86 ?= "c994390cfa28339cbc1ec3b56eeec83a5fa75bb7"
SRCREV_machine_qemux86-64 ?= "c994390cfa28339cbc1ec3b56eeec83a5fa75bb7"
SRCREV_machine ?= "c994390cfa28339cbc1ec3b56eeec83a5fa75bb7"
SRCREV_meta ?= "80b4b5110dca5184b57e85f1e775fb006a2e85ad"

SRC_URI = "git://git.yoctoproject.org/linux-yocto-3.4.git;protocol=git;bareclone=1;branch=${KBRANCH},${KMETA};name=machine,meta"

LINUX_VERSION ?= "3.4.36"

PR = "${INC_PR}.3"
PV = "${LINUX_VERSION}+git${SRCPV}"

KMETA = "meta"

COMPATIBLE_MACHINE = "qemuarm|qemux86|qemuppc|qemumips|qemux86-64"

# Functionality flags
KERNEL_FEATURES_append = " features/netfilter/netfilter.scc"
KERNEL_FEATURES_append_qemux86=" cfg/sound.scc"
KERNEL_FEATURES_append_qemux86-64=" cfg/sound.scc"
KERNEL_FEATURES_append_qemux86=" cfg/paravirt_kvm.scc"
KERNEL_FEATURES_append = " ${@bb.utils.contains("TUNE_FEATURES", "mx32", " cfg/x32.scc", "" ,d)}"

先查看下 linux-yocto_3.4.bb require 的 recipes-kernel/linux/linux-yocto.inc

DESCRIPTION = "Yocto Kernel"
SECTION = "kernel"
LICENSE = "GPLv2"

LIC_FILES_CHKSUM = "file://COPYING;md5=d7810fab7487fb0aad327b76f1be7cd7"

INC_PR = "r4"

# A KMACHINE is the mapping of a yocto $MACHINE to what is built
# by the kernel. This is typically the branch that should be built,
# and it can be specific to the machine or shared
# KMACHINE = "UNDEFINED"

LINUX_KERNEL_TYPE ?= "standard"

# KMETA ?= ""
KBRANCH ?= "master"
KMACHINE ?= "${MACHINE}"
SRCREV_FORMAT ?= "meta_machine"

LINUX_VERSION_EXTENSION ?= "-yocto-${LINUX_KERNEL_TYPE}"

do_patch[depends] = "kern-tools-native:do_populate_sysroot"

addtask kernel_configme before do_configure after do_patch

# Pick up shared functions
inherit kernel
inherit kernel-yocto
require linux-dtb.inc

B = "${WORKDIR}/linux-${MACHINE}-${LINUX_KERNEL_TYPE}-build"

do_install_append(){
        if [ -n "${KMETA}" ]; then
                rm -rf ${STAGING_KERNEL_DIR}/${KMETA}
        fi
}

# extra tasks
addtask kernel_link_vmlinux after do_compile before do_install
addtask validate_branches before do_patch after do_kernel_checkout
addtask kernel_configcheck after do_configure before do_compile
这里inherit kernel, inherit kernel-yocto, 在meta/class 里面可以看到 kernel.class ,kernel-yocto.class 文件,里面有kernel 公共的base 函数。

http://blog.chinaunix.net/uid-7652108-id-2047290.html

bitbake parse 的机理分析:

    Bitbake 这个 task execute tool的第一步就是parsing,也就是对BBFILES所定义的变量的内容,也就是bbfiles 进行读取data,然后分析,相关数据进行缓存(cache)
 
     出发是从 conf/bitbake.conf 这个文件开始,这个文件本身的内容处理是一部分,另外一部分是这个文件会include一些其它的conf文件,比如很重要的 conf/local.conf,分析处理的数据都会写到 CACHEDATA的文件中(bb_cache.dat) ,当然前提是你开启了cache机制
    处理完conf文件后,就是bbclass文件,这个时候是从 class/base.bbclass开始,然后分析其inherit的一些class。
    这两部分数据很重要,后来所有的bbfile 都依赖于这些conf 文件和class文件,当然这些文件中的变量也会实施到所有的bb file上,所以只要这些conf文件和class文件有改变,那么bitbake 就会重新parse
   那么,到现在,就开始进入parsing bb file了,当然第一次肯定要parse所有的bbfile了,这部分是最耗时的,第一次完了之后,那么就可以利用cache机制来判断是否重新parse了
 
   上面就是bitbake version 1版本parsing的机制,理解了code背后的故事,才能让我们更游刃有余地工作

膜拜这个大神,早在07年就把 bitbake 一个package 的流程分析的这么透彻了:

http://blog.chinaunix.net/uid-7652108-id-2047247.html


在 linux-yocto.inc 文件中 inherit kernel-yocto, 来看下 class/kernel-yocto.bbclass , 分析一下do_kernel_configme 这个函数。添加log语句打印一下这个函数中的一些变量。 ${S} 表示linux source code 的目录, ${B} 表示linux build 目录。

在 linux_source_code_path/meta/scripts/configme 执行这个shell 脚本。

do_kernel_configme[dirs] = "${S} ${B}"
do_kernel_configme() {
        echo "[INFO] doing kernel configme"
        export KMETA=${KMETA}

        if [ -n ${KCONFIG_MODE} ]; then
                configmeflags=${KCONFIG_MODE}
        else
                # If a defconfig was passed, use =n as the baseline, which is achieved
                # via --allnoconfig
                if [ -f ${WORKDIR}/defconfig ]; then
                        configmeflags="--allnoconfig"
                fi
        fi

        cd ${S}
        PATH=${PATH}:${S}/scripts/util
        configme ${configmeflags} --reconfig --output ${B} ${LINUX_KERNEL_TYPE} ${KMACHINE}
        if [ $? -ne 0 ]; then
                echo "ERROR. Could not configure ${KMACHINE}-${LINUX_KERNEL_TYPE}"
                exit 1
        fi

        #qc added log
        echo "qc variables value: ${S} ${B} ${KMETA} ${KCONFIG_MODE} ${WORKDIR} ${LINUX_KERNEL_TYPE} ${KMACHINE}."
        echo "# Global settings from linux recipe" >> ${B}/.config
        echo "CONFIG_LOCALVERSION="\"${LINUX_VERSION_EXTENSION}\" >> ${B}/.config
}

configme 这个 shell 脚本完成 生成.config 文件的工作:

# This is factored out into a function because for a given branch,
# there may be more than one user (i.e. big endian, little endian,
# or BSPs that use the same branch but differ only in kernel configs)
run_board_config()
{
    # Can't set these until we've unwound the checkpoint and have meta data.
    KVER=`cat ./$META_DIR/cfg/kernel-*cache/kver|sed 's/^v//'`

    # Look for standard defines, with compatibility fallbacks
    KARCH=`grep KARCH $SCC | awk '{print $3}'`
    KPROFILE=`grep KMACHINE $SCC | awk '{print $3}'`
    KTYPE=`grep KTYPE $SCC | awk '{print $3}'`

    META=./$META_DIR/meta-series
    META_ALT=./$META_DIR/cfg/scratch/`basename $SCC .scc`-meta

    BUILD_DIR=$out_dir
    CFGFILE=$machine-$target-config-$KVER
    kgit-meta -v -k $META
    if [ $? != 0 ]; then
        echo Error running the meta series for collecting config data
        return 1
    fi

    KTGT=`get_branch_name $META`
    mkdir -p ./$META_DIR/cfg/$KTGT
    if [ $? != 0 ]; then
        echo Failed to mkdir ./$META_DIR/cfg/$KTGT for config data
        return 1
    fi

    frags=`cat $META_DIR/cfg/$KTGT/config_frag.txt | sed 's%\(^.*$\)%'$META_DIR/cfg'\1%'`
    pre_config -l $META_DIR/cfg/$KTGT/ $frags > $META_DIR/cfg/$KTGT/config.log 2>&1

    # remove any old assembled debug fragments
    rm -f $BUILD_DIR/.tmp.config*

    merge_frags=`cat $META_DIR/cfg/$KTGT/config_frag.txt | sed 's%\(^.*$\)%'$META_DIR/cfg'\1.sanitized%'`
    ARCH=$KARCH O=$BUILD_DIR merge_config.sh $allnoconfig -d $merge_frags  \
                                      > $META_DIR/cfg/$KTGT/merge_log.txt 2>&1

    mv $BUILD_DIR/.tmp.config* $META_DIR/cfg/$KTGT/$CFGFILE
    if [ $? != 0 ]; then
        echo creation of pre-processed config data failed
        return 1
    fi

    # break the merge log down into parts that can be processed later
    grep -A2 "^Value of" $META_DIR/cfg/$KTGT/merge_log.txt > $META_DIR/cfg/$KTGT/redefinition.txt
    grep -A2 "^Value requested" $META_DIR/cfg/$KTGT/merge_log.txt > $META_DIR/cfg/$KTGT/mismatch.txt

    echo "[INFO] Pre-processed cfg file $CFGFILE created."

 

拼接一系列的meta 目录下的config_frag 文件,最终通过 merge_config.sh 生成 .config 文件。

/yocto_build/tmp/work/qemuppc-poky-linux/linux-yocto/3.4.36+gitAUTOINC+80b4b5110dca5184b57e85f1e775fb006a2e85ad_ddbc382cbc45a009e9ce17f7d448fcfd050ab5fc-r4.3/linux/meta/cfg/standard/qemuppc/config_frag.txt

这个就是要处理的所有 config 的片段。

可以参考 config.log 和 merge_log.txt

/yocto_build/tmp/work/qemuppc-poky-linux/linux-yocto/3.4.36+gitAUTOINC+80b4b5110dca5184b57e85f1e775fb006a2e85ad_ddbc382cbc45a009e9ce17f7d448fcfd050ab5fc-r4.3/linux/meta/cfg/standard/qemuppc/config.log

/yocto_build/tmp/work/qemuppc-poky-linux/linux-yocto/3.4.36+gitAUTOINC+80b4b5110dca5184b57e85f1e775fb006a2e85ad_ddbc382cbc45a009e9ce17f7d448fcfd050ab5fc-r4.3/linux/meta/cfg/standard/qemuppc/merge_log.txt

至此,就知道 yocto 编译 linux kernel 的 .config 是从哪来的了!!!

了解更多关于 yocto kernel config 可以 git clone git://git.yoctoproject.org/yocto-kernel-cache,查看 00-README 文档。

http://blog.csdn.net/fmddlmyy/article/details/325403

在 bitbake 中加入 print log语句,方便调试:

    parser.add_option("-B", "--bind", help = "The name/address for the bitbake server to bind to",
               action = "store", dest = "bind", default = False)
    parser.add_option("", "--no-setscene", help = "Do not run any setscene tasks, forces builds",
               action = "store_true", dest = "nosetscene", default = False)
    options, args = parser.parse_args(sys.argv)
    
    #qc test code
    print "qc test code: options:", options
    print "qc test code: args", args

    configuration = BBConfiguration(options)
    configuration.pkgs_to_build.extend(args[1:])


©️2020 CSDN 皮肤主题: 大白 设计师:CSDN官方博客 返回首页