jenkins+docker进行coverity检查自动构建

使用docker技术管理Jenkins服务器。避免多次部署需要重复安装的重复工作，且可以方便迁移到新的服务器。这么好的技术，忽略多可惜。因此，我做了相关的部分工作。

前期准备：

找了一个朋友用来制作镜像的环境，找到了一份基本操作系统的镜像。

检查我们的自动构建的环境已经安装了docker客户端，我只需要把制作好的镜像pull下来，就可以启动容器。

我的工作：

1.利用基本操作系统的镜像制作安装有coverity客户端软件的镜像并推送到制品库

从制作镜像的机器某目录放了三个文件：

*_docker.sh， *-Linux-MAIN.V5.05.F7-x86_64.dvd.iso，rpmlistfile

第一个文件是执行制作镜像的shell脚本，脚本中增加拷贝不能安装的内容。第二个文件时基础操作系统的镜像，第三个文件时是增加 rpmlistfile 安装内容，里面是一系列安装文件的名字。

执行./*_docker.sh --iso *-Linux-MAIN.V5.05.F7-x86_64.dvd.iso --file rpmlistfile 可以生成新的镜像文件。对rpmlistfile里的安装文件名字根据报错增减（如bash grep gcc make dos2unix unix2dos等等）。

镜像生成成功后启动容器

docker run -it repository:tag /bin/bash

将coverity的安装文件拷贝进入容器

docker cp cov-analysis-linux64-2019.06.sh 460f6b5be1e5:/home

在容器内安装coverity,安装过程参照

https://blog.csdn.net/qq_33163046/article/details/104038678

在安装步骤第一步时，出现报错

java.lang.Error: Probable fatal error:No fonts found.

可能是字体库存在缺失。返回制作镜像的步骤，在rpmlistfile里增加dejavu* ,fontconfig ,ttmkfdir 可以解决。

修改镜像名字

docker commit -a "xxz" -m "*_coverity" 3e0492adad98  the_address_in_artnj.cop.com.cn_coverity:*_coverity/v1_0307

推送到制品库，推送钱需要登陆制品库，并确认自己有权限

docker push the_address_in_artnj.cop.com.cn_coverity:*_coverity/v1_0307

2.在jenkinsfile文件里编写代码构建coverity全量检查与增量检查的函数

在jenkinsfile.txt编写了两个函数，coverity_incre()函和coverity_all()函数。函数里调用了两个shell脚本,coverity.sh脚本和incre_coverity.sh脚本

jenkinsfile.txt增加内容

//coverity增量扫描容器
def coverity_incre(){
    env.starttimebd = new Date().format('yyyy-MM-dd HH:mm:ss');
    env.bgcoverity="bg2"
    my_bgcoverity=1
    def coverity_verifyci=0
    node(node_name){
       withEnv(["work_space=${env.CODE}"]){
           ws("${work_space}"){
               sh '''
                    ls -l;
                    rm -rf coverity_verifyci.log;
                    rm -rf html;
                    rm -rf analyze_failure.flag;
                    docker logout docker.artnj.cop.com.cn
                    docker login * -u $user -p $passwd
                    docker_image=*
                    docker pull $docker_image
                    docker run --rm -i\\
                    -v /root/.ssh/:/root/.ssh/ \\
                    -v $work_space:/home/proto \\
                    -e "PROJECT_DIR=/home/coverity" \\
                    -w /home \\
                    $docker_image /bin/bash -c \\
                    "cd proto;
                    cp -rf buildpath/src buildpath/lib buildpath/bin buildpath/make ../coverity;
                    cp -rf protocol/pipe protocol/codediff ../coverity;
                    cd /home/coverity;ls -la;
                    HOME=/home/coverity;\\
                    export PATH=$PATH:$HOME/bin:/usr/local/cov-analysis-linux64-2019.06/bin; \\
                    cat codediff;
                    cat codediff|grep -E '.c$|.h$|.cpp$|.hpp$'>codediff_cache||true;
                    cat codediff_cache;
                    if [ -s codediff_cache ];then
                        pwd;source ./.bash_profile;cd src;cp -rf ../pipe/incre_coverity.sh .;chmod 755 -Rf *.sh;dos2unix *.sh;./setproenv_64.sh;\\
                        ls -l;./incre_coverity.sh proto 10.47.0.0 STREAM_NAME;                                                                       
                    fi
                    if [ -e /home/coverity/coverity_verifyci/analyze_failure.flag ];then
                        cp /home/coverity/coverity_verifyci/analyze_failure.flag  /home/proto/log;\\
                        chmod 777 /home/proto/log/analyze_failure.flag;
                    fi
                    if [ -s /home/coverity/coverity_verifyci/coverity_verifyci.log ];then
                        cp /home/coverity/coverity_verifyci/coverity_verifyci.log  /home/proto/log;\\
                        chmod 777 /home/proto/log/coverity_verifyci.log;\\
                        cov-format-errors --dir /home/coverity/coverity_verifyci/ --html-output /home/coverity/coverity_verifyci/html --title proto -x;
                    fi
                    if [ -s /home/coverity/coverity_verifyci/html/index.html ];then
                        cp -rf /home/coverity/coverity_verifyci/html  /home/proto;\\
                        chmod -R 777 /home/proto/html;
                    fi"
                '''
                if (fileExists("${work_space}/log/analyze_failure.flag"))
                {
                    coverity_verifyci=2
                }
                if (fileExists("${work_space}/log/coverity_verifyci.log"))
                {
                    coverityResult = readFile("${work_space}/coverity_verifyci.log")
                    if(coverityResult.contains("Defect only exists locally"))
                    {
                        println "coverity scan error!!!"
                        coverity_verifyci=1
                        publishHTML([allowMissing: false, alwaysLinkToLastBuild: true, keepAll: true, reportDir: "${work_space}/log", reportFiles: 'coverity_verifyci.log', reportName: 'Coverity incre Scan Result'])
                        if (fileExists("${work_space}/html/index.html"))
                        {
                            publishHTML([allowMissing: false, alwaysLinkToLastBuild: true, keepAll: true, reportDir: "${work_space}/html", reportFiles: 'index.html', reportName: 'Coverity incre Scan Result.html'])		               
                        }
                    }
                }
            }
        }
    }
    if(coverity_verifyci==1)
    {
        error  "coverity scan error"
    }
    else if (coverity_verifyci==2)
    {
        error  "coverity analyze error"	
    }
    else
    {
        env.bgcoverity="bg1"
        my_bgcoverity=0
        println "coverity scan successful"
    }
}
//coverity全量扫描容器
def coverity_all(){
    env.starttimebd = new Date().format('yyyy-MM-dd HH:mm:ss');
    my_bgcoverity=1
    env.bgcoverity=1
    node(node_name){
       withEnv(["work_space=${env.CODE}"]){
           ws("${work_space}"){
                sh '''
                    rm -rf index.html;
                    rm -rf html;
                    ls -l;
                    docker logout docker.artnj.cop.com.cn
                    docker login * -u $art_user -p $art_passwd					
                    docker_image=*
                    docker pull $docker_image
                    docker run --rm -i\\
                    -v /root/.ssh/:/root/.ssh/ \\
                    -v $work_space:/home/proto \\
                    -e "PROJECT_DIR=/home/coverity" \\
                    -w /home \\
                    $docker_image /bin/bash -c \\
                    "ls -la;cd proto;ls -l;
                    cp -rf buildpath/src buildpath/lib buildpath/bin buildpath/make ../coverity;
                    cp -rf protocol/pipe ../coverity;
                    cd /home/coverity;
                    ls -la;
                    HOME=/home/coverity;\\
                    export PATH=$PATH:$HOME/bin:/usr/local/cov-analysis-linux64-2019.06/bin; \\
                    source ./.bash_profile;cd src;cp -rf ../pipe/coverity.sh .;chmod 755 -Rf *.sh;dos2unix *.sh;./setproenv_64.sh;\\
                    ls -l;./coverity.sh proto 10.47.0.0 STREAM_NAME;
                    if [ -s /home/coverity/coverity_proto/html/index.html ];then
                        cp -rf  /home/coverity/coverity_proto/html  /home/proto;
                        chmod -R 777 /home/proto/html;
                    fi"
                '''
                if (fileExists($work_space/coverity_coverity/html/index.html"))
                {
                    publishHTML([allowMissing: false, alwaysLinkToLastBuild: true, keepAll: true, reportDir: "${work_space}/html", reportFiles: 'index.html', reportName: 'Coverity Scan Result'])                    
                    println 'coverity publishHTML over'
                }
            }
        }
    }
    env.bgcoverity = 0
    println 'coverity is successful'
    my_bgcoverity=0
}

coverity.sh

#!/bin/bash

set -x

BRANCH_VERSION=$1
COV_SERVER_ADDR=$2
SOURCE_STREAM=$3

midd_dir="$HOME/coverity_${BRANCH_VERSION}"

if [ -d "$midd_dir" ]; then
    rm -rf $midd_dir
fi
html_path="${midd_dir}/html"

OLDIFS="$IFS"
IFS=$'\n'
stream_info=(`cov-manage-im --host ${COV_SERVER_ADDR} --user username --password password --mode streams --show --name "${SOURCE_STREAM}"`)
    if [ ${#stream_info[*]} -eq 1 ];then
        echo "================================================================================="
        echo "|| error : no $source_stream stream in Coverity Platform ||"
        echo "================================================================================="
        exit 1
    fi
IFS="$OLDIFS"

cov-configure --gcc
cov-build --dir $midd_dir make Clean Install
cov-analyze --dir $midd_dir --all \
--rule \
--enable-constraint-fpp \
--enable-callgraph-metrics \
--enable-fnptr \
--enable-virtual \
--enable USER_POINTER \
--enable DC.STRING_BUFFER \
--enable ENUM_AS_BOOLEAN \
--enable UNENCRYPTED_SENSITIVE_DATA \
--enable WEAK_GUARD \
--enable WEAK_PASSWORD_HASH \
--enable HARDCODED_CREDENTIALS \
--enable AUDIT.SPECULATIVE_EXECUTION_DATA_LEAK \
--enable INTEGER_OVERFLOW \
--enable MIXED_ENUMS \
--enable RISKY_CRYPTO \
--enable COM.ADDROF_LEAK \
--enable COM.BSTR.ALLOC \
--enable COM.BSTR.BAD_COMPARE \
--enable COM.BSTR.NE_NON_BSTR \
--enable FLOATING_POINT_EQUALITY \
--enable VCALL_IN_CTOR_DTOR \
--strip-path=$HOME
if [ $? -ne 0 ]; then
    echo "error: coverity analyze failure"
    exit 1
fi
cov-format-errors --dir $midd_dir --html-output $html_path --lang zh-cn --title name -x
cov-commit-defects --dir $midd_dir --host ${COV_SERVER_ADDR}  --user username --password password --stream $SOURCE_STREAM
if [ $? -ne 0 ]; then
    echo "error: coverity commit failure"
    exit 1
fi

incre_coverity.sh

#!/bin/bash

set -x

BRANCH_VERSION=$1
COV_SERVER_ADDR=$2
SOURCE_STREAM=$3

midd_dir="$HOME/coverity_verifyci"

if [ -d "$midd_dir" ]; then
    rm -rf $midd_dir
fi
html_path="${midd_dir}/html"


cov-configure --gcc
cov-build --dir $midd_dir make

cd ..
filediff=`cat codediff_cache`
echo $filediff

cov-run-desktop --dir $midd_dir --all \
--rule \
--enable-constraint-fpp \
--enable-callgraph-metrics \
--enable-fnptr \
--enable-virtual \
--enable USER_POINTER \
--enable DC.STRING_BUFFER \
--enable ENUM_AS_BOOLEAN \
--enable UNENCRYPTED_SENSITIVE_DATA \
--enable WEAK_GUARD \
--enable WEAK_PASSWORD_HASH \
--enable HARDCODED_CREDENTIALS \
--enable AUDIT.SPECULATIVE_EXECUTION_DATA_LEAK \
--enable INTEGER_OVERFLOW \
--enable MIXED_ENUMS \
--enable RISKY_CRYPTO \
--enable COM.ADDROF_LEAK \
--enable COM.BSTR.ALLOC \
--enable COM.BSTR.BAD_COMPARE \
--enable COM.BSTR.NE_NON_BSTR \
--enable FLOATING_POINT_EQUALITY \
--enable VCALL_IN_CTOR_DTOR \
--strip-path=$HOME \
--ignore-uncapturable-inputs true --host ${COV_SERVER_ADDR}  --user user  --password password  --stream "${SOURCE_STREAM}" \
--reference-snapshot latest   --present-in-reference false  --impact-regex "Medium|High"  --lang zh-cn --text-output $midd_dir/coverity_verifyci.log $filediff
if [ $? -ne 0 ]; then
    echo "error: coverity analyze failure"
    exit 1
fi
if [ -e $midd_dir/coverity_verifyci.log ]; then
	chmod  777 $midd_dir/coverity_verifyci.log
fi

3.测试

镜像拉取日志

16:50:56 [Coverity] v1_0307: Pulling from *

16:50:56 [Coverity] a62007c53c34: Pulling fs layer

16:50:56 [Coverity] ea8e416f481e: Pulling fs layer

16:51:05 [Coverity] a62007c53c34: Verifying Checksum

16:51:05 [Coverity] a62007c53c34: Download complete

16:51:18 [Coverity] ea8e416f481e: Verifying Checksum

16:51:18 [Coverity] ea8e416f481e: Download complete

16:51:30 [Coverity] a62007c53c34: Pull complete

16:53:21 [Coverity] ea8e416f481e: Pull complete

16:53:21 [Coverity] Digest: sha256:e387fea821bbc9ae851ea51bf7b90963bd1906d2e6ac15dc43340bbb4fb83fc6

执行过程中用“docker ps -a”可以观察到容器启动执行结束后主动销毁。

观察到执行成功日志

17:19:30 [Coverity] New snapshot ID 15628 added.

检查确实有最新coverity检查提交的记录。

在全量检查和增量检查结束后，jenkins界面会出现

和

点击进去可以看到检测结果。