【gitlab+jenkins+docker】第三节 docker版jenkins安装和配置

前言

使用现成的jenkins容器能够快速搭建jenkins服务，且管理起来很方便，具有容灾、可移植等优点(放轻松，操作过程与上节神似，有些步骤如果操作过就不用再操作了)

路径创建

为了避免不必要的错误，建议数据存放路径与我这里保持一致：

[root@JackRoy ~]# mkdir -p /opt/dockers/docker_tool_jenkins/online
[root@JackRoy ~]# mkdir -p /opt/dockers/docker_tool_jenkins/gitlab_data
[root@JackRoy ~]# cd /opt/dockers/docker_tool_jenkins/gitlab_data

封装docker启动命令

如果已经跟随上一节操作过，这里就不用再操作了，如果没有跟随上一节操作，直接便捷/usr/bin/dockerutils输入：

[root@JackRoy online]# vim /usr/bin/dockerutils
#!/bin/bash
# Settle issue (e.g.: SELINUX) of when mount host directory, it has no privilege to access. 
chcon -Rt svirt_sandbox_file_t $dir > /dev/null 2>&1
container_name=
TOP_PID=$$
trap 'exit 127' TERM
gate_way=`route | grep 'default' | awk '{print $2}'`
code=0
find_name(){
    _container_name=`cat $dir/docker-compose.yml  | grep container_name | awk -F ':' '{print $2}' |  sed  's/^[       ]*//g'  | sed  's/[ ]*$//g'`
    _container_name=${_container_name//\"/}
    container_name=${_container_name//\'/}
    if [ x$container_name == x ]
       then
         echo -e "\033[31mERROR : please set container_name in your docker-compose.yml file \033[0m"
         kill -s TERM $TOP_PID
    fi
}
check(){
    _host_name=`cat $dir/docker-compose.yml  | grep hostname | awk -F ':' '{print $2}' |  sed  's/^[       ]*//g'  | sed  's/[ ]*$//g'`
    _host_name=${_host_name//\"/}
    host_name=${_host_name//\'/}
echo "-------------------------------------------------"$host_name
    if [ x$host_name == x ]
       then
         echo -e "\033[31mERROR : hostname not set !  \033[0m"
		 code=1
    fi	
   _ip=`cat $dir/docker-compose.yml  | grep ipv4_address | awk -F ':' '{print $2}' |  sed  's/^[       ]*//g'  | sed  's/[ ]*$//g'`
   _ip=${_ip//\"/}
   ip=${_ip//\'/}
   if [ $1 = y ] ; then
      fail=`ping $ip -c 3 -s 1 -W 1 | grep "100% packet loss" | wc -l`
      if [ $fail -eq 0 ] 
         then
         echo -e "\033[31mERROR : address is already in use \033[0m"
         code=1
      fi
    
   fi
   if [ $code -ne 0 ] ; then  kill -s TERM $TOP_PID  ;fi 

}
up(){
    check n
    docker-compose -f $dir/docker-compose.yml up -d
    start
}
reup(){
   docker rm -f $container_name
   up
}
start(){
   docker start $container_name
}

stop(){
   docker stop $container_name
}

restart(){
   docker stop $container_name
   sleep 3
   docker start $container_name
}
usage(){
     echo -e " Usage : \n \t $0 [ usage ] |  < up | reup | start | stop | restart > "
     echo -e "\t ============= nat docker ==============="
     echo -e "\t up : create container and start app"
     echo -e "\t reup : re-create container and start app" 
     echo -e "\t start : start container and app"
     echo -e "\t stop : stop container and app"
     echo -e "\t restart : restart container and app" 
     kill -s TERM $TOP_PID
}
if [ "$1"x = ""x ] || [ "$1"x = "usage" ] ; then
   usage
fi
find_name
$@ 
exit $?

创建启动配置文件

进入/opt/dockers/docker_tool_jenkins/online目录下：

[root@JackRoy online]# cd /opt/dockers/docker_tool_jenkins/online

编辑deploy_comm.sh并输入：

#!/bin/bash
dir=$(cd -P -- "$(dirname -- "$0")" && pwd -P)
.  dockerutils $@

编辑docker-compose.yml并输入：

version: '2.2'
services:
  jenkins:
    image: jenkins/jenkins:lts 
    container_name: jenkins
    hostname: jenkins
    volumes:
      - "../jenkins_home:/var/jenkins_home1"
    cpus: 1
    mem_limit: 2G
    environment:
      - TZ=Asia/Shanghai
      - LANG=en_US.UTF-8
    ports:
      - "8082:8080"
      - "50000:50000"
    networks:
      aidata_network:
        ipv4_address: 10.111.113.50

networks:
  aidata_network:
    external: true

启动

在/opt/dockers/docker_tool_gitlab/online路径下，执行启动命令

# deploy_comm.sh 支持 reup up stop多种命令
[root@JackRoy online]# sh deploy_comm.sh reup
Error response from daemon: No such container: jenkins
-------------------------------------------------jenkins
WARNING: Found orphan containers (gitlab) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
Pulling jenkins (jenkins/jenkins:lts)...
Trying to pull repository docker.io/jenkins/jenkins ... 
lts: Pulling from docker.io/jenkins/jenkins
844c33c7e6ea: Pull complete
ada5d61ae65d: Pull complete
f8427fdf4292: Pull complete
f025bafc4ab8: Pull complete
67b8714e1225: Pull complete
64b12da521a3: Pull complete
2e38df533772: Pull complete
b1842c00e465: Pull complete
b08450b01d3d: Pull complete
2c6efeb9f289: Pull complete
0805b9b9cdc4: Pull complete
f129619fc383: Pull complete
cd27f3a82cdf: Pull complete
f31251f493ed: Pull complete
2c902f1f4dfa: Pull complete
2fe1d2cb7aab: Pull complete
908723de775f: Pull complete
54aa3899e429: Pull complete
f48cf8764dc1: Pull complete
Digest: sha256:d5069c543e80454279caacd13457d012fb32c5229b5037a163d8bf61ffa6b80b
Status: Downloaded newer image for docker.io/jenkins/jenkins:lts
Creating jenkins ... done
jenkins

执行启动命令以后我们进入docker容器，执行命令“cp -r /var/jenkins_home/* /var/jenkins_home1”：

[root@JackRoy online]# docker exec -it jenkins bash
jenkins@jenkins:/$ cp -r /var/jenkins_home/* /var/jenkins_home1
jenkins@jenkins:/$ exit
exit

退出后修改docker-compose.yml文件（/var/jenkins_home1改为/var/jenkins_home）：

version: '2.2'
services:
  jenkins:
    image: jenkins/jenkins:lts 
    container_name: jenkins
    hostname: jenkins
    volumes:
      - "../jenkins_home:/var/jenkins_home"     # 修改位置
    cpus: 1
    mem_limit: 2G
    environment:
      - TZ=Asia/Shanghai
      - LANG=en_US.UTF-8
    ports:
      - "8082:8080"
      - "50000:50000"
    networks:
      aidata_network:
        ipv4_address: 10.111.113.50

networks:
  aidata_network:
    external: true

在/opt/dockers/docker_tool_gitlab/online路径下，执行启动命令

# deploy_comm.sh 支持 reup up stop多种命令
[root@JackRoy online]# sh deploy_comm.sh reup
jenkins
-------------------------------------------------jenkins
WARNING: Found orphan containers (gitlab) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
Creating jenkins ... done
jenkins

页面配置

访问url：

http://10.20*.4.5*:8082/login?from=%2F

效果如下：

初始提示我们需要从本地复制密码，OK，我们按照提示路径打开jenkins容器寻找初始密码：

[root@JackRoy ~]# docker exec -it jenkins bash
jenkins@jenkins:/$ cat /var/jenkins_home/secrets/initialAdminPassword
138d763d6d374274ae5b490fcb442017
jenkins@jenkins:/$ exit
[root@JackRoy ~]# 

将初始密码粘贴后，可以看到：

这里我们选择“安装推荐的插件”就好，里面涵盖了包括git等我们常用的插件。
接着便是等待这些插件的安装，如果网络不卡，则时间不会很久，等待的尽头是：

按照提示，我们创建用户，同时设定密码，点击“保存并完成”后，进入下一步：

直接点击“保存并完成”，接着就可以开始使用jenkins了。

安装Maven插件

因为很多项目我们是使用了maven仓库来管理项目中的依赖，因此，在jenkins上完成项目打包是依赖maven插件的，依次点击Manage Jenkins（系统管理）、Manage Pligins（插件管理）

点击“可选插件”，在右上角输入搜索“Maven Integration”，勾选后点击“直接安装”：

稍等片刻即安装成功：

凭据配置

在添加凭据之前，我们需要做一件事，就是在我们的jenkins docker容器内生成公钥（配置至git中）与私钥（配置至jenkins全局凭据中），这样才能组成一把完整的锁与钥匙的关系，操作明细如下：

# 命令1
[root@JackRoy ~]# docker exec -it jenkins bash
# 命令2
jenkins@jenkins:/$ cd  
# 命令3
jenkins@jenkins:~$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/var/jenkins_home/.ssh/id_rsa): 
/var/jenkins_home/.ssh/id_rsa already exists.
# 输入y
Overwrite (y/n)?y
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /var/jenkins_home/.ssh/id_rsa.
Your public key has been saved in /var/jenkins_home/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:hsfnzzVqlYtR6SpI0Xbc5e6VE3IPPZtDjYvXdR9WluA jenkins@jenkins
The key is randomart image is:
+---[RSA 2048]----+
|             .. .|
|            .  .+|
|         . . E Bo|
|       o. o o.*B*|
|      . So.. +=*%|
|       o.o  o ***|
|       . ..  *oo+|
|        . .o+o...|
|           o+    |
+----[SHA256]-----+
# 命令4 顺便做一下与宿主机的免密登录：
jenkins@jenkins:~$ ssh-copy-id -p 22 -i .ssh/id_rsa.pub root@10.20*.4.5* 
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: ".ssh/id_rsa.pub"
The authenticity of host '10.20*.4.5* (10.20*.4.5*)' can't be established.
ECDSA key fingerprint is SHA256:tsX+R0v8maofjGIJc01fcGidEsEqG0I6q6tzm3FP+2o.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
# 输入宿主机roto 密码
root@10.20*.4.5*'s password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh -p '22' 'root@10.20*.4.5*'"
and check to make sure that only the key(s) you wanted were added.
# 命令5 测试一下免密登录
jenkins@jenkins:~$  ssh root@10.20*.4.5*
Last login: Mon Dec 30 09:58:17 2019 from 10.43.0.72
# 命令6
[root@JackRoy ~]# exit
logout
Connection to 10.20*.4.5* closed.
# 命令7
jenkins@jenkins:~$ cd .ssh/
# 命令8
jenkins@jenkins:~/.ssh$ cat id_rsa
-----BEGIN RSA PRIVATE KEY-----
MIIEpAIBAAKCAQEAxYUI7MtQs7uaMRGTEbJ7oknmIstbRWRwsVPKoE33RrOcUTwp
v2m1aOw7KwAGceP2jq+RrRtt58adLYVvvENML+JeCITdZ4kuZ3UeWuzvJvk2b7hv
UvAz1wnup9ZONRSNUqDwY/0KjIM4ZDiQpkUo1pUbqiAvBO5n8G/j3AhDa0NEU0hw
MdT0PbAUfVzvTQ7OMLmDhkHHpuZ33rZz4OqOTm7IXVWlT9dBmVtsymDAezKNt+SA
ySo80RTsP2OkobuzH6gRWY0Mf9Fpt4VUe4sYPa7Vs1QeH6Bv6iJB49Rp5tmZTojv
bKAWu7IDmimK9jAwge59MDSOc9lj+NBQadCUwwIDAQABAoIBAQC61Fla0f2l5F+2
5i68YJz9JvVeGAhfHOBTezSAaP7qi13ioQvEGJMVic11oPJ3q+89i+g5W2acA4wM
0NidtPJNoMrdN//MvT99cJSj4NVdggDxr/zs8HO4hZaOfvE+2A/NBpobme/udPhU
JAT+oRA92JBwXTzcZz09ejiHmfFbdH2WpZeGMdhOndSSY0F71U7RXiDShC8o/TMw
+6Db5eu3OTU4lJ1yKWMOBzz/Dtsr8d5gqq32GueWtkTLd6N8fH7pnuZuNVrBBGkq
b68MKRW6ophjzoYFtKV26wXaGdBPSSSLUIZ+MsF3H5RlSJgagSFp035NGPTv6jRv
qlnHUJKRAoGBAOaDWIpuxLymDV/hiDnU+Q/veNYj6JOSYFDvcus27yhv9SDjlwF
Y1rc0SDehliRzkIeSLHUJNJaij9P9bkhGeVDbW7N+eTwu6LU0+RuFH+DNM2ITNH7K
AgQgAbmd3mT+UYa0f3UJ63Gk7QU1dLQC/DI+2Mayh0kNpXIl2uIXzyUnAoGBANtb
0DjV+t+QjhD+OEBKOziLEPm3SPa78687yEADnU8LjPMrgKxd0nYNta6wzWqcq8GBnc3p
jqfMtWAzkSWyYBOVG786867ZW9JovwSdWgFCpRiHEc090T/gh3BUWNm1jkPH0/cgnszDRqdzQ
i2yIFR/Q2XIrh+DGLy2miE+7u5Wgne2BQwOE+y0FAoGBAOIu60gaXGznYWzNDp1m
nwiOCxLzKS0HSFTQVYFuwcYbqJEr7Ro0909aJ11W4tUvw/9QQmTOVFUajUFpjVtjhzzf
+dX/B8HeTbNiC/ESY4Q/YgJBFzQayB3ItaW6ijbJ0xih+vkETzRWYXIHdJLx2utv
KmtbA3YkY16pRzf0MQw8dINtAoGABd7tMbg4YKdjxfyn0MCr2RWPvL81CrV35I60
NRtgCFoPtY1FRreTNzuwI7xRtHodmRc5JDCMYii/J70gaSXCa4mGcJRbxYXlRESu
XCif00pEs6PGjSLvDCq8aBD38MSLwUcrlBoIRrO2PiIAHvnu5U/hshCaMqT71RFy
k7OJbWkCgYATyutuyfA7yHLaS613zm0dkqXtA+5xku/AE4jh0XolaBVODL27U+ohuzpm4
wTltcZIGF+q6H3VAGS17pa5haLgMSwdh9hpFo5S/MIgaW+6++YBffFwKU4VkfYD1
kwoO4z21O1myVSZoPdilqwGp7EcFy+Z/K8xigJlHOber1WJ7y8YY9w==
-----END RSA PRIVATE KEY-----
# 命令9
jenkins@jenkins:~/.ssh$ cat id_rsa.pub 
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFhQjsy1Czu5oxMRsnuiSeYiy1tFZHCxU8gTfdGs5xRPCm/abVo7DsrAAZx4/aOr5GtG23nxp0thW+8Q0wv4l4IhN1niS5ndR5a7O8m+TZvuG9S8DPXCe6n1k41FI1SoPB0j/QqMgzhkOJCmRSjWlRuqIC8E7mfwb+PcCENrQ0RTSHAx1PQ9sBR9XO9NDs4wuYOGQcem5nfetnPg6o985ObshdVaVP10GZW2zKYMB7Mo235IDJKjzRFOw/Y6Shu7MfqBFZjQx/0Wm3hVR7ixg9rtWzVB4foG/qIkHj1Gnm2ZlOiO9soBa7sgOaKYr2MDCB7n0wNI5z2WP40FBp0JTD jenkins@jenkins
jenkins@jenkins:~/.ssh$ exit
exit
[root@JackRoy  ~]#

拿到生成的私钥(注意是私钥，就是id_rsa，打印信息长的那个)以后，返回页面，在左边菜单栏点击进入凭据：

选择添加凭据：

在页面中选择并填入相关信息（这里面我没有选中的都是你自己可以随便填的，不会影响你使用）：

点击确定：
凭据添加成功，这时候不要忘记把jenkins docker容器的公钥添加进gitlab!
添加公钥至gitlab操作明细我上一节有介绍：第二节 docker版jenkinsan安装和配置

配置全局工具（jdk和maven）

这里笔者没有选择使用自动安装，具体原因是因为jdk现在下载有诸多不方便（需要账号），以及会留下不必要的坑，这里我自己准备了jdk和maven，放入了宿主机的/opt/dockers/docker_tool_jenkins/jenkins_home/路径下的java目录和maven目录，注意，一定要放在这个路径或者其子路径下，因为之前我们在docker compose里面配置了路径映射，该路径下的文件会被映射至docker容器中(java和maven目录是笔者创建的，里面分别存放了jdk1.8和maven3.0.4)：

[root@JackRoy   opt]# cd /opt/dockers/docker_tool_jenkins/jenkins_home/
[root@JackRoy   jenkins_home]# ll
total 84
-rw-r--r--  1 develop develop  477 Dec 30 10:09 com.cloudbees.hudson.plugins.folder.config.AbstractFolderConfiguration.xml
-rw-r--r--  1 develop develop 1647 Dec 30 10:13 config.xml
-rw-r--r--  1 develop develop  100 Dec 27 19:33 copy_reference_file.log
-rw-r--r--  1 develop develop 3468 Dec 30 11:42 credentials.xml
-rw-r--r--  1 develop develop  156 Dec 27 19:34 hudson.model.UpdateCenter.xml
-rw-r--r--  1 develop develop  370 Dec 30 10:09 hudson.plugins.git.GitTool.xml
-rw-------  1 develop develop 1712 Dec 27 19:30 identity.key.enc
drwxr-xr-x  3 root    root      25 Dec 23 16:01 java
-rw-r--r--  1 develop develop    7 Dec 30 10:13 jenkins.install.InstallUtil.lastExecVersion
-rw-r--r--  1 develop develop    7 Dec 30 10:13 jenkins.install.UpgradeWizard.state
-rw-r--r--  1 develop develop  181 Dec 30 10:13 jenkins.model.JenkinsLocationConfiguration.xml
-rw-r--r--  1 develop develop  171 Dec 27 19:30 jenkins.telemetry.Correlator.xml
drwxr-xr-x  2 develop develop    6 Dec 27 19:30 jobs
drwxr-xr-x  3 develop develop   18 Dec 27 19:30 logs
drwxr-xr-x  3 root    root      31 Dec 23 16:01 maven
-rw-r--r--  1 develop develop  907 Dec 27 19:34 nodeMonitors.xml
drwxr-xr-x  2 develop develop    6 Dec 27 19:30 nodes
drwxr-xr-x 80 develop develop 8192 Dec 30 10:23 plugins
-rw-r--r--  1 develop develop   64 Dec 27 19:30 secret.key
-rw-r--r--  1 develop develop    0 Dec 27 19:30 secret.key.not-so-secret
drwx------  4 develop develop 4096 Dec 30 11:42 secrets
-rw-r--r--  1 develop develop 7152 Dec 27 19:30 tini_pub.gpg
drwxr-xr-x  2 develop develop 4096 Dec 30 10:09 updates
drwxr-xr-x  2 develop develop   23 Dec 27 19:30 userContent
drwxr-xr-x  3 develop develop   54 Dec 30 10:12 users
drwxr-xr-x 11 develop develop 4096 Dec 27 19:34 war
drwxr-xr-x  2 develop develop    6 Dec 30 10:09 workflow-libs
[root@JackRoy   jenkins_home]#

增加一下这两个目录的权限（必须要做）：

[root@JackRoy   jenkins_home]# chmod 777 -R java/
[root@JackRoy   jenkins_home]# chmod 777 -R maven/

做好这些准备后，进入页面中的Global Tool Configuration（全局配置）：

在JDK选项点击选择新增JDK：

录入必要的两项信息（“/var/jenkins_home/java/jdk1.8.0_111”就是宿主机路径“/opt/dockers/docker_tool_jenkins/jenkins_home/java/jdk1.8.0_111”在docker中的映射路径，如果你通晓这一切就可以自由决定这些路径的映射关系，如果是半生不熟，那最好跟笔者这里保持一致，避免不必要的错误）：

添加maven的操作也类似：

信息录入完成后，点击保存。

创建JOB

这里一定要保证jenkins docker的公钥成功添加至gitlab的key中（公钥的获得方法在上一段有介绍），我这里贴一下效果图：

在首页面，选择新建job:

录入作业名，选择构建模式后点击确定（我这里选择了之前上传至gitlab中的项目中的一个子项目为例）：

录入项目地址信息和监控的分支：

选择构建方式（多种模式可选，比较实用就是触发远程构建和分支变更执行两种，这里做测试我就选了定时轮询）和jdk：

录入pom的位置，选择构建后的操作为shell脚本(先不要纠结这部分脚本)：

点击保存不要运行（虽然它会自己运行），因为后续的自动化脚本还没有部署，所以运行不会成功的。

后记

到这里基本的框架就算搭起来了，后续的生成docker容器，启动项目都被我做成自动化的了，目前web项目和server项目均支持，让我们能够实现一键上传，轻松部署（涵盖了历史项目缓存功能，以便能够回滚），已下这段逻辑主要是定义变量，部署逻辑封装在编辑的脚本里，这里贴一下jenkins的打包逻辑（server类项目的模板）：

再往后的小节就是用来讲自动化部署的代码的了。

跳转

第一节 基础环境介绍与准备
第二节 docker版gitlab安装和配置
第三节 docker版jenkins安装和配置
第四节 docker版mysql安装和配置
第五节 docker信息明细表设计与实现（含收集脚本）
第六节 docker容器自动化生成脚本
第七节 自动生成docker容器并启动项目