python—android编译中build_image.py分析

最近在学习python，基础知识学习了一遍。我的学习习惯是：了解了基础语法后，去阅读开源代码（掌握优秀程序员的一些写代码习惯和手法），最后结合项目实践，实践的过程对某些细节进行深化。
想起android编译系统中有一些python脚本，下面对build_image.py进行简单分析。

编译系统中调用build_image.py

生成系统镜像的target在build\core\Makefile中，

#----------build\core\Makefile-------------
# $(1): output file
define build-systemimage-target
  @echo "Target system fs image: $(1)"
  @mkdir -p $(dir $(1)) $(systemimage_intermediates) && rm -rf $(systemimage_intermediates)/system_image_info.txt
  $(call generate-userimage-prop-dictionary, $(systemimage_intermediates)/system_image_info.txt, skip_fsck=true)
  $(hide) PATH=$(foreach p,$(INTERNAL_USERIMAGES_BINARY_PATHS),$(p):)$$PATH \
      ./build/tools/releasetools/build_image.py \
      $(TARGET_OUT) $(systemimage_intermediates)/system_image_info.txt $(1)
endef

其中，可以看出来主要是借助了build_image.py去生成镜像，/build/tools/releasetools/build_image.py $(TARGET_OUT) $(systemimage_intermediates)/system_image_info.txt $(1) ， 有三个参数，一个是作为输入的文件目录，system_image_info.txt，和最终生成的镜像名，例如system.img。

build_image.py分析

由于上面makefile中是在shell中直接调用build_image.py(不是import 模块)，所以__name__为__main__，会直接调用main函数，

#参数格式必须为3个
# $(TARGET_OUT) system_image_info.txt output_file
def main(argv):
  if len(argv) != 3:
    print __doc__
    sys.exit(1)

  in_dir = argv[0]
  glob_dict_file = argv[1]
  out_file = argv[2]

  #将system_image_info.txt中的内容保存到字典中，例如{'fs_type':'ext4',.....}
  glob_dict = LoadGlobalDict(glob_dict_file)
  #basename获取文件名(不包括路径)，这里为system.img
  image_filename = os.path.basename(out_file)
  mount_point = ""
  if image_filename == "system.img":
    mount_point = "system"
  elif image_filename == "userdata.img":
    mount_point = "data"
  elif image_filename == "cache.img":
    mount_point = "cache"
  elif image_filename == "vendor.img":
    mount_point = "vendor"
  else:
    print >> sys.stderr, "error: unknown image file name ", image_filename
    exit(1)
  #print >> 
  #the first expression after the >> must evaluate to a “file-like” object, 
  #输入为system_image_info.txt内容字典和system
  image_properties = ImagePropFromGlobalDict(glob_dict, mount_point)
  if not BuildImage(in_dir, image_properties, out_file):
    print >> sys.stderr, "error: failed to build %s from %s" % (out_file, in_dir)
    exit(1)

#当被直接python build_image.py执行时 __name__为__main__
#当被其他import时，__name__是模块的名字，build_image
#argv[0]是脚本名
if __name__ == '__main__':
  main(sys.argv[1:])

其中，system_image_info.txt一个例子如下所示，一些参数及其值，

fs_type=ext4
system_size=1288491008
userdata_size=5835325440
cache_fs_type=ext4
cache_size=268435456
extfs_sparse_flag=-s
selinux_fc=out/target/product/msm8916_64/root/file_contexts
verity=true
verity_key=build/target/product/security/verity
verity_signer_cmd=out/host/linux-x86/bin/verity_signer
system_verity_block_device=/dev/block/bootdevice/by-name/system
skip_fsck=true

LoadGlobalDict函数如下，将system_image_info.txt中的内容保存到字典中，例如{‘fs_type’:’ext4’,…..}

def LoadGlobalDict(filename):
  """Load "name=value" pairs from filename"""
  d = {}
  f = open(filename)
  #对文件对象的迭代
  #文件中#不处理，是注释
  #strip去除字符串两头，不包括内部的空格
  #split将字符串分割成序列 str.split(sep=None, maxsplit=-1) 
  #If maxsplit is given, at most maxsplit splits are done (thus, the list will have at most maxsplit+1 elements). 
  #>>>'1,2,3'.split(',', maxsplit=1)
  #['1', '2,3']
  #所以用=分割，只有=前面和后面2个元素
  for line in f:
    line = line.strip()
    if not line or line.startswith("#"):
      continue
    k, v = line.split("=", 1)
    d[k] = v
  f.close()
  return d

ImagePropFromGlobalDict函数，从全局字典(system_image_info.txt中有些东西不是都有用)取出需要的东西，返回一个新字典。

def ImagePropFromGlobalDict(glob_dict, mount_point):
  """Build an image property dictionary from the global dictionary.

  Args:
    glob_dict: the global dictionary from the build system.
    mount_point: such as "system", "data" etc.
  """
  d = {}

  def copy_prop(src_p, dest_p):
    if src_p in glob_dict:
      d[dest_p] = str(glob_dict[src_p])
 #extfs_sparse_flag=-s
 #skip_fsck=true
 #selinux_fc=out/target/product/msm8916_64/root/file_contexts
 #将common_props属性中的值保存到新建的字典d中，
  common_props = (
      "extfs_sparse_flag",
      "mkyaffs2_extra_flags",
      "selinux_fc",
      "skip_fsck",
      )
  for p in common_props:
    copy_prop(p, p)

  #添加'mount_point':'system'
  d["mount_point"] = mount_point
  if mount_point == "system":
    #'fs_type':'ext4'
    #'partition_size':'1288491008'
    copy_prop("fs_type", "fs_type")
    copy_prop("system_size", "partition_size")
  elif mount_point == "data":
    copy_prop("fs_type", "fs_type")
    copy_prop("userdata_size", "partition_size")
  elif mount_point == "cache":
    copy_prop("cache_fs_type", "fs_type")
    copy_prop("cache_size", "partition_size")
  elif mount_point == "vendor":
    copy_prop("vendor_fs_type", "fs_type")
    copy_prop("vendor_size", "partition_size")

  return d

最后调用BuildImage函数，其实就是组了一个命令，然后执行该命令(mkuserimg.sh -s in_dir out_file ext4 system 1288491008 out/target/product/msm8916_64/root/file_contexts)。

def BuildImage(in_dir, prop_dict, out_file):
  """Build an image to out_file from in_dir with property prop_dict.

  Args:
    in_dir: path of input directory.
    prop_dict: property dictionary.
    out_file: path of the output image file.

  Returns:
    True iff the image is built successfully.
  """
  build_command = []
  #字典的get方法，如果键不存在，没有任何异常，而得到None
  fs_type = prop_dict.get("fs_type", "")
  run_fsck = False
  #文件系统以ext开头
  #这里是ext4，走这里,其实就是组一个要执行的命令 mkuserimg.sh -s in_dir out_file ext4 system 1288491008 out/target/product/msm8916_64/root/file_contexts
  if fs_type.startswith("ext"):
    build_command = ["mkuserimg.sh"]
    if "extfs_sparse_flag" in prop_dict:
      build_command.append(prop_dict["extfs_sparse_flag"])
      run_fsck = True
    build_command.extend([in_dir, out_file, fs_type,
                          prop_dict["mount_point"]])
    if "partition_size" in prop_dict:
      build_command.append(prop_dict["partition_size"])
    if "selinux_fc" in prop_dict:
      build_command.append(prop_dict["selinux_fc"])
  else:
    build_command = ["mkyaffs2image", "-f"]
    #split用于将字符串分割成list，没有参数表示用空格分割
    if prop_dict.get("mkyaffs2_extra_flags", None):
      build_command.extend(prop_dict["mkyaffs2_extra_flags"].split())
    build_command.append(in_dir)
    build_command.append(out_file)
    if "selinux_fc" in prop_dict:
      build_command.append(prop_dict["selinux_fc"])
      build_command.append(prop_dict["mount_point"])

  #执行该命令mkuserimg.sh -s in_dir out_file ext4 system
  exit_code = RunCommand(build_command)
  if exit_code != 0:
    return False
# >>> os.path.dirname('c:\\Python\\a.txt') 返回目录
# 'c:\\Python'
# >>> os.path.basename('c:\\Python\\a.txt') 返回文件名
# 'a.txt'
# os.path.join连接目录与文件名或目录

  #skip_fsck=true，不走这里 
  if run_fsck and prop_dict.get("skip_fsck") != "true":
    # Inflate the sparse image
    unsparse_image = os.path.join(
        os.path.dirname(out_file), "unsparse_" + os.path.basename(out_file))
    inflate_command = ["simg2img", out_file, unsparse_image]
    exit_code = RunCommand(inflate_command)
    #This function is semantically identical to unlink().
    #remove和unlink一样
    if exit_code != 0:
      os.remove(unsparse_image)
      return False

    # Run e2fsck on the inflated image file
    e2fsck_command = ["e2fsck", "-f", "-n", unsparse_image]
    exit_code = RunCommand(e2fsck_command)

    os.remove(unsparse_image)

  return exit_code == 0

其中，RunCommand其实就类似于c语言中的system()，执行一段命令，获取返回值，用popen实现的。

def RunCommand(cmd):
  """ Echo and run the given command

  Args:
    cmd: the command represented as a list of strings.
  Returns:
    The exit code.
  """
  print "Running: ", " ".join(cmd)
  #创建子进程
  p = subprocess.Popen(cmd)
  #等待子进程执行完
  p.communicate()
  return p.returncode

最终其实是调用了mkuserimg.sh脚本，脚本核心是调用make_ext4fs 应用程序

#system\extras\ext4_utils\mkuserimg.sh

function usage() {
cat<<EOT
Usage:
mkuserimg.sh [-s] SRC_DIR OUTPUT_FILE EXT_VARIANT MOUNT_POINT SIZE [FILE_CONTEXTS]
EOT
}

echo "in mkuserimg.sh PATH=$PATH"

ENABLE_SPARSE_IMAGE=
if [ "$1" = "-s" ]; then
  ENABLE_SPARSE_IMAGE="-s"
  shift
fi

if [ $# -ne 5 -a $# -ne 6 ]; then
  usage
  exit 1
fi

SRC_DIR=$1
if [ ! -d $SRC_DIR ]; then
  echo "Can not find directory $SRC_DIR!"
  exit 2
fi

OUTPUT_FILE=$2
EXT_VARIANT=$3
MOUNT_POINT=$4
SIZE=$5
FC=$6

case $EXT_VARIANT in
  ext4) ;;
  *) echo "Only ext4 is supported!"; exit 3 ;;
esac

if [ -z $MOUNT_POINT ]; then
  echo "Mount point is required"
  exit 2
fi

if [ -z $SIZE ]; then
  echo "Need size of filesystem"
  exit 2
fi

if [ -n "$FC" ]; then
    FCOPT="-S $FC"
fi

#核心是make_ext4fs 应用程序
MAKE_EXT4FS_CMD="make_ext4fs $ENABLE_SPARSE_IMAGE $FCOPT -l $SIZE -a $MOUNT_POINT $OUTPUT_FILE $SRC_DIR"
echo $MAKE_EXT4FS_CMD
$MAKE_EXT4FS_CMD
if [ $? -ne 0 ]; then
  exit 4
fi

而make_ext4fs 应用程序具体在system\extras\ext4_utils中生成，makefile如下，其中makefile中有两个make_ext4fs的module，一个是host(BUILD_HOST_EXECUTABLE)，一个是target的(BUILD_EXECUTABLE)，这个命令是在编译机调用，也就是host上而不是target上。

include $(CLEAR_VARS)
LOCAL_SRC_FILES := make_ext4fs_main.c
LOCAL_MODULE := make_ext4fs
LOCAL_STATIC_LIBRARIES += \
    libext4_utils_host \
    libsparse_host \
    libz
ifeq ($(HOST_OS),windows)
  LOCAL_LDLIBS += -lws2_32
else
  LOCAL_STATIC_LIBRARIES += libselinux
  LOCAL_CFLAGS := -DHOST
endif
include $(BUILD_HOST_EXECUTABLE)