合patch脚本使用说明
背景
开发hisiv551过程中,海思一个月会发一个基于整个源码的patch包,其中文件很多,同时konka开发的代码中有很多客制化的地方,因此不能覆盖合并,因此,每次合patch很慢,由于代码多,手工合代码久了难免出错。基于多次合patch,发现海思给的patch的基础版本和konka做修改的基础版本一致,因此只需要手工合konka客制化的文件,其他文件就可以覆盖更新。基于这个思想,写了合并patch脚本。
需求
- 筛选出并备份需要合入patch目录下客制化修改了patch。根据客制化修改时打的标记“konka patch”,如果某次修改没有打标记,那么合hisilicon patch时会冲掉客制化修改。筛选出的客制化修改了的文件在执行脚本的一个*_bak目录中。
- 备份的文件和patch中的文件对比,删除没有被修改的客制化文件,减少手动合并的文件数。如果选择和公板代码对比合并,此步骤无意义。
- 拷贝patch目录中的文件至合并的目录(merger dir)
- 删除patch中没有的文件或目录。此步骤仅用于和公板完整代码对比合并。具体使用说明详见后文。
配置
一些明确客制化的目录和不是合patch执行者熟悉的模块,可以配置不合,让相应熟悉那些模块的同事合并。脚本内容详见后面。
配置如下:
EXCLUDE_DIR="(\<konka\>|\<KKhi3751v551\>|\<KKhi3751v551_board\>|\<wifi\>|\<bluetooth\>|\<pub\>)"
只能配置不需要合的目录名称,配置的目录名称以shell的正则表达式中标记单词开头“<”和结尾“>”括起来,不同目录之间以shell中的“|”分隔。自动合完patch后,客制化的目录手动合并。
配置客制化修改标记
KEYWORDS_PATTERN="(konka[ ]+patch)"
konka patch,两个单词间没有空格限制
使用
使用脚本前,先执行hisilicon中的delete.sh,然后再运行脚本,脚本使用说明如下:
Usage: ./patch_merge.sh merge_dir [patch_dir] [bPatch[0|1]]
merge_dir 需要合并patch的目录路径
patch_dir patch目录路径
bPatch 合并patch,patch是否为公板完整代码的目录,如果是,则值为0;反之为1.如果值需要为0,可以不传,默认值为0.该值在“需求”中的第四步使用。
例:
./patch_merge.sh Android_V551
仅筛选出被客制化修改了的文件
./patch_merge.sh Android_V551 Android_HS
使用hisilicon公板代码合并,并在merge_dir中删除公板代码中没有的文件和目录
./patch_merge.sh Android_V551 Android_HS 1
使用hisilicon公板代码合并,不删除在merge_dir中有但公板代码中没有的文件和目录
./patch_merge.sh Android_V551/device/hisilicon/bigfish/sdk
使用海思给的patch合并,只是公板代码中需要修改的文件
脚本
#!/bin/bash
## patch_merge.sh
function _help()
{
cat <<EOF
Usage: ./patch_merge.sh merge_dir [patch_dir] [bPatch[0|1]]
merge_dir target dir for merge patch
patch_dir patch dir from hisilicon
bPatch value is 0 or 1,default is 1.if use hisilicon entire code to merge or some files
that konka code have and don't exist in hisilicon demo code need to be delete,the value is set to be 0;
if use hisilicon patch code to patch,it need to set the value to be 1;
example:
./patch_merge.sh Android_V551
only select files that are modified by patch
./patch_merge.sh Android_V551 Android_HS
patch use hisilicon demo code
./patch_merge.sh Android_V551/device/hisilicon/bigfish/sdk
patch/sdk 1 patch use part of code
EOF
}
set -e
if [ $# -lt 1 ];then
_help
exit 2
fi
bPatch=1
if [ $# -eq 3 ];then
bPatch=$3
fi
EXCLUDE_DIR="(\<konka\>|\<wifi\>|\<bluetooth\>|\<pub\>|\<KKhi3751v810\>|\<out\>|\<ccache\>)"
KEYWORDS_PATTERN="(konka[ \t]+patch)"
DEBUG=false
PER_MAX_SIZE_M=30
MERGE_DIR=$(echo $1 | sed -e 's:/$::;s:^./::')
if [ $# -ge 2 ];then
PATCH_DIR=$(echo $2 | sed -e 's:/$::;s:^./::')
fi
merge_root=${MERGE_DIR%%/*}
BACKUP_DIR=${merge_root}_bak${MERGE_DIR#*${merge_root}}
echo -e "\e[36m--------------"
echo "bPatch $bPatch"
echo "DEBUG $DEBUG"
echo "EXCLUDE_DIR $EXCLUDE_DIR"
echo "KEYWORDS_PATTERN $KEYWORDS_PATTERN"
echo "MERGE_DIR $MERGE_DIR"
echo "PATCH_DIR $PATCH_DIR"
echo "BACKUP_DIR $BACKUP_DIR"
echo -e "--------------\e[0m "
if [ x"$MERGE_DIR" == x -o ! -d $MERGE_DIR ];then
echo -e "\e[31merror: ***error: MERGE_DIR is not set\e[0m"
_help
return 2
fi
if [ x"$PATCH_DIR" != x ];then
if [ ! -d $PATCH_DIR ];then
echo -e "\e[31merror: ***error: PATCH_DIR:$PATCH_DIR is not exist\e[0m"
_help
return 2
fi
fi
TOP=$(pwd)
if [ ! -d $TOP/tmp ];then
mkdir -p $TOP/tmp
fi
if [ -d $BACKUP_DIR ];then
rm -fr $BACKUP_DIR
fi
# 查找客制化文件,返回文件列表
function find_backup_files()
{
local work_dir=$1
local only_file=0
local files=
if [ x`echo $1 | grep ==` != x ];then
only_file=1
work_dir=${work_dir%==*}
fi
(
cd $work_dir
if [ $only_file -eq 1 ];then
files=$(find -depth -mindepth 1 -maxdepth 1 -type f -exec grep -riHE "$KEYWORDS_PATTERN" {} \; | awk -F: '{print $1}' | sort | uniq)
else
files=$(grep -riE "$KEYWORDS_PATTERN" | awk -F: '{print $1}' | sort | uniq)
fi
for f in $files;do echo $work_dir/${f#*\./};done
)
}
# 根据客制化的文件列表,做好备份
function backup_OEM_files()
{
if [ $# -eq 0 ];then
[ "$DEBUG" = "true" ] && echo "No files are backuped"
return 0
fi
echo -e "\e[36m------------ backup"
for f in $@;do echo $f;done
echo -e "---------------------\e[0m"
for file in $@
do
file=${file##*${MERGE_DIR#*/}/}
local file_dir=${file%/*}
local src_file=${MERGE_DIR}/$file
local dst_file=${BACKUP_DIR}/$file
echo -e "\e[36m+++++++++++++++++++"
echo file : $file
echo src_file : $src_file
echo dst_file : $dst_file
echo file_dir : $file_dir
mkdir -p "$BACKUP_DIR/$file_dir"
cp -fv $src_file $dst_file
echo -e "+++++++++++++++++++\e[0m"
done
}
# 1、判断当前目录占用总空间是否大于PER_MAX_SIZE_M,如果是,转到第二步;反之,寻找被客制化的文件并备份
# 2、遍历当前目录下目录,小于PER_MAX_SIZE_M的目录查找并备份;大于PER_MAX_SIZE_M的目录,递归到步骤1
# 3、目录处理完,在当前目录查找并备份客制化文件
function backup_OEM_dir()
{
local ROOT_DIR=$1
local CHECK_DIR=$2
[ "$DEBUG" = "true" ] && echo
[ "$DEBUG" = "true" ] && echo ROOT_DIR : $ROOT_DIR
[ "$DEBUG" = "true" ] && echo CHECK_DIR: $CHECK_DIR
local size=`du -hm --max-depth=0 $ROOT_DIR | awk '{print $1}'`
echo $size $ROOT_DIR ...
[ x"$CHECK_DIR" != x ] && echo CHECK_DIR: $CHECK_DIR
if [ $size -gt $PER_MAX_SIZE_M ];then
for sub_dir in `cd $ROOT_DIR;ls -F | grep "/$" | sed -e "s:/$::" | grep -vE $EXCLUDE_DIR`
do
if [ x"$CHECK_DIR" != x ];then
[ "$DEBUG" = "true" ] && echo -e "\n"----- check $CHECK_DIR/$sub_dir
test ! -d $CHECK_DIR/$sub_dir && continue
fi
local sub_size=`du -hm --max-depth=0 "$ROOT_DIR/$sub_dir" | awk '{print $1}'`
[ "$DEBUG" = "true" ] && echo ----- $ROOT_DIR/$sub_dir:$sub_size
if [ $sub_size -gt $PER_MAX_SIZE_M ];then
if [ x"$CHECK_DIR" == x ];then
CHECK_SUB_DIR=
else
CHECK_SUB_DIR=$CHECK_DIR/$sub_dir
fi
backup_OEM_dir "$ROOT_DIR/$sub_dir" "$CHECK_SUB_DIR"
else
# backup files of $ROOT_DIR/$sub_dir and its sub dirs
[ "$DEBUG" = "true" ] && echo "backup files of $ROOT_DIR/$sub_dir and its sub dirs"
backup_OEM_files $(find_backup_files $ROOT_DIR/$sub_dir)
fi
done
# only do backup files of $ROOT_DIR
[ "$DEBUG" = "true" ] && echo "only do backup files of $ROOT_DIR"
backup_OEM_files $(find_backup_files ${ROOT_DIR}==)
else
# backup files in current and sub dirs
[ "$DEBUG" = "true" ] && echo "backup files in current and sub dirs"
backup_OEM_files $(find_backup_files $ROOT_DIR)
fi
return 0
}
# 如果konka客制化的文件不在海思修改的文件中,则从备份文件中删除,不用合并
# 如果选择使用海思整个公板代码合并,这一步基本无意义
function delete_no_modify_files()
{
local SRC_DIR=$1
local TARGET_DIR=$2
rm -f $TOP/tmp/tmp_patch_files
rm -f $TOP/tmp/tmp_backup_files
rm -f $TOP/tmp/tmp_delete_files_no_modify.txt
for sub_dir in $(cd $TARGET_DIR;ls -F | grep "/$" | sed -e "s:/$::")
do
echo -e "handle \e[36mSRC->$SRC_DIR/$sub_dir ...\e[0m"
(cd $SRC_DIR;if test -d $sub_dir;then find $sub_dir -type f | sort >> $TOP/tmp/tmp_patch_files;fi)
echo -e "handle \e[36mtarget->$TARGET_DIR/$sub_dir ...\e[0m"
(cd $TARGET_DIR;if test -d $sub_dir;then find $sub_dir -type f | sort >> $TOP/tmp/tmp_backup_files;fi)
done
if [ -f $TOP/tmp/tmp_backup_files -a -f $TOP/tmp/tmp_patch_files ];then
comm -2 -3 $TOP/tmp/tmp_backup_files $TOP/tmp/tmp_patch_files > $TOP/tmp/tmp_delete_files_no_modify.txt
# 1.删除没有被海思修改的文件
echo -e "\e[36mdelete files are not modified ...\e[0m"
for rm_file in `cat $TOP/tmp/tmp_delete_files_no_modify.txt`
do
if [ ! -f $SRC_DIR/$rm_file ];then
rm -fv $TARGET_DIR/$rm_file
fi
done
# 2.delete empty directory
echo -e "\e[36mdelete empty directories ...\e[0m"
while [ x"$(find $TARGET_DIR -empty -type d)" != x ]
do
find $TARGET_DIR -empty -type d | while read line
do
rm -frv "$line"
done
done
fi
}
function patch_ODM_dir()
{
if [ ! -d $BACKUP_DIR ];then
# 3.patch ODM entire dir to OEM target dir
cp -fa $PATCH_DIR/* $MERGE_DIR/
echo -e "\e[36mNo konka patch exist at \"$BACKUP_DIR\"\e[0m"
return 0
fi
find $PATCH_DIR -type d | grep -vE $EXCLUDE_DIR | while read dir
do
local tmpDir=$(echo ${dir#*${PATCH_DIR}} | sed "s:^/::")
if [ ! -d $MERGE_DIR/$tmpDir ];then
mkdir -p $MERGE_DIR/$tmpDir
fi
find $dir -depth -maxdepth 1 -type f | while read tmpFile
do
# 不拷贝软连接文件
if [ x"$(ls -l "$tmpFile" | grep -v "^l")" = x ];then
continue
fi
# 删除merge目录的软连接文件
if [ -L $MERGE_DIR/$tmpDir/$(basename $tmpFile) ];then
rm -f $MERGE_DIR/$tmpDir/$(basename $tmpFile)
fi
echo -e "patch files \e[36m$tmpFile\e[0m"
cp -f "$tmpFile" $MERGE_DIR/$tmpDir
done
done
# echo -e "patch files under $PATCH_DIR to \e[36m$MERGE_DIR\e[0m"
# find $PATCH_DIR -depth -max-depth 1 -exec cp -f --parents {} $MERGE_DIR/ \;
}
function delete_no_use_files()
{
if [ $# -ne 2 ];then
echo "error: params error"
return 2;
fi
local SRC_DIR=$1
local TARGET_DIR=$2
local depth=1
while [ 1 -eq 1 ]
do
local dirs=`cd $TARGET_DIR;find -depth -mindepth $depth -maxdepth $depth -type d | grep -vE $EXCLUDE_DIR`
[ "$DEBUG" = true ] && echo
echo depth : $depth
echo $dirs
let depth++
if [ x"$dirs" == x ];then
break
fi
for dir in $dirs
do
# 删除非konka客制化且公板没有的目录
dir=$(echo $dir | sed -e 's:^\./\(.*\):\1:g')
if [ ! -d $SRC_DIR/$dir ];then
echo -e "\e[37mDelte dir:$TARGET_DIR/$dir\e[0m"
rm -frv $TARGET_DIR/$dir
continue
fi
# 生成存在目录的文件列表
echo check files of $dir
(cd $SRC_DIR; find $dir -depth -maxdepth 1 -type f | sort > $TOP/tmp/tmp_src_files)
(cd $TARGET_DIR;find $dir -depth -maxdepth 1 -type f | sort > $TOP/tmp/tmp_target_files)
comm -2 -3 $TOP/tmp/tmp_target_files $TOP/tmp/tmp_src_files > $TOP/tmp/tmp_delete_files_no_use.txt
# 删除公板没有且不是konka客制化的文件
while read rm_file
do
if -f [ $TARGET_DIR/"$rm_file" ];then
rm -fv $TARGET_DIR/"$rm_file"
fi
done < $TOP/tmp/tmp_delete_files_no_use.txt
done
done
if [ 1 -eq 0 ];then
if [ -f $TOP/tmp/tmp_src_files -a -f $TOP/tmp/tmp_target_files ];then
comm -2 -3 $TOP/tmp/tmp_target_files $TOP/tmp/tmp_src_files > $TOP/tmp/tmp_delete_files_no_use.txt
# 删除公板没有且不是konka客制化的文件
while read rm_file
do
if -f [ $TARGET_DIR/"$rm_file" ];then
rm -fv $TARGET_DIR/"$rm_file"
fi
done < $TOP/tmp/tmp_delete_files_no_use.txt
fi
fi
}
if [ ! -f $TOP/tmp/patch_record.txt ];then
echo 1 > $TOP/tmp/patch_record.txt
elif [ x"$(cat $TOP/tmp/patch_record.txt)" = x"end" ];then
rm -f $TOP/tmp/patch_record.txt
echo 1 > $TOP/tmp/patch_record.txt
fi
case $(cat $TOP/tmp/patch_record.txt) in
1)
# 1.backup OEM modify files to backup dir
# rm -fr $BACKUP_DIR
echo -e "\e[31mBegin backuping ...\e[0m"
backup_OEM_dir ${MERGE_DIR} ${PATCH_DIR}
echo -e "\e[31mBackup end\e[0m\n"
echo 2 > $TOP/tmp/patch_record.txt
;;
esac
# 2.delete files are not change in backup dir
if [ x"$PATCH_DIR" != x ];then
case $(cat $TOP/tmp/patch_record.txt) in
2)
find ${PATCH_DIR} -type d -name ".svn" | sed -e "s/^/\'/g;s/$/\'/g" | xargs rm -frv
find ${PATCH_DIR} -name ".git*" | sed -e "s/^/\'/;s/$/\'/" | xargs rm -frv
if [ -d $BACKUP_DIR ];then
echo -e "\e[31mdelete_no_modify_dir ...\e[0m"
delete_no_modify_files $PATCH_DIR $BACKUP_DIR
echo -e "\e[31mdelete_no_modify_dir end\e[0m\n"
fi
echo 3 > $TOP/tmp/patch_record.txt
;&
3)
# 3.patch ODM files to OEM target dir
echo -e "\e[31mpatch_ODM_dir ...\e[0m"
patch_ODM_dir
echo -e "\e[31mpatch_ODM_dir end\e[0m\n"
echo 4 > $TOP/tmp/patch_record.txt
;&
4)
if [ $bPatch -eq 0 ];then
# 4.delete some files in OEM target dir,but thers art not in ODM target dir
echo -e "\e[31mdelete_no_use_files ...\e[0m"
delete_no_use_files $PATCH_DIR $MERGE_DIR
echo -e "\e[31mdelete_no_use_files end\e[0m\n"
fi
echo "end" > $TOP/tmp/patch_record.txt
;;
esac
fi
echo ------end