Jenkins & CICD
使用Jenkins实现完整的CI/CD pipeline:
- 程序员修改源码
- git commit & push 提交修改
- 触发CICD pipeline
- Jenkins将源码打包成jar,封装入docker
- 自动运行测试,生成报告
- 构建好的版本上传到Docker Hub
- 最新的版本部署到production生产环境
实验环境搭建
Docker in Docker
Docker In Docker(也称为dind
)允许开发人员在已经运行的Docker容器中运行新的Docker容器,常用场景:
- 支持CI/CD管道
- 创建沙盒容器环境
CI/CD管道支持
在本实验中,CI/CD pipeline要依赖在容器中运行的Jenkins。而容器中的Jenkins需要执行docker指令。
沙盒环境
DinD可以将Docker与主机环境隔离开来,防止污染主机环境。
在Docker容器中运行Docker的方法
实现Docker in Docker有多种方法
- 使用docker:dind标签。此方法需要额外的权限管理。
- 挂载主机Docker套接字。将主机的/var/run/docker.sock套接字映射到docker内部。这样进程可以docker容器内部向外部主机的docker engine发送指令。这种方法也被称为Docker Outside of Docker (DooD)。
- 使用Sysbox。Sysbox 是一个开源的专用容器运行时,可以在不需要特权模式的情况下嵌套容器。
本次实验我们使用最简单的Docker套接字挂载方法实现DinD。
挂载主机Docker套接字
修改套接字所有权
ls -al /var/run/docker.sock
sudo chown $UID:$UID /var/run/docker.sock
创建新的docker环境
mkdir ~/pipeline
cd ~/pipeline
touch Dockerfile
docker exec -it jenkins bash
cat /etc/os-release
# NAME="Debian GNU/Linux"
目前的jenkins docker机遇Debian环境,安装对应的docker
FROM jenkins-ansible
USER root
# Install Docker
RUN apt-get update && \
apt-get -y install apt-transport-https \
ca-certificates \
curl \
gnupg2 \
software-properties-common && \
curl -fsSL https://download.docker.com/linux/$(. /etc/os-release; echo "$ID")/gpg > /tmp/dkey; apt-key add /tmp/dkey && \
add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/$(. /etc/os-release; echo "$ID") \
$(lsb_release -cs) \
stable" && \
apt-get update && \
apt-get -y install docker-ce
# Compose
RUN curl -L "https://github.com/docker/compose/releases/download/1.22.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose && chmod +x /usr/local/bin/docker-compose
RUN usermod -aG docker jenkins
USER jenkins
修改compose,加入安装了docker的jenkins docker
docker-compose.yml
version: '3'
services:
jenkins:
container_name: jenkins
image: jenkins/docker
build:
context: pipeline
ports:
- "8080:8080"
volumes:
- $PWD/jenkins_home:/var/jenkins_home
- /var/run/docker.sock:/var/run/docker.sock
networks:
- net
remote_host:
container_name: remote-host
image: remote-host
build:
context: centos
networks:
- net
db_host:
container_name: mysql
image: mysql:5.7
environment:
- "MYSQL_ROOT_PASSWORD=1234"
volumes:
- $PWD/db_data:/var/lib/mysql
networks:
- net
web:
container_name: web
image: ansible-web
build:
context: web
ports:
- "80:80"
volumes:
- $PWD/web/html:/var/www/html
networks:
- net
networks:
net:
验证与docker engine的通信
docker compose build
docker compose up -d
docker exec -it jenkins bash
docker ps
# CONTAINER ID IMAGE ....
搭建 pipeine
获取项目代码
将之前的项目代码复制到新的文件夹,修改pom.xml
的maven-compiler-plugin
版本
cd ~/pipeline
cp -r ~/workspace/maven java-app
rm -rf java-app/.git*
vim java-app/pom.xml
#<artifactId>maven-compiler-plugin</artifactId>
# <version>3.10.1</version>
获取maven容器
docker pull maven
docker images | grep maven
使用maven构建代码,查看打包好的jar文件
docker run --rm -v $PWD/java-app:/app -v /root/.m2/:/root/.m2/ -w /app maven mvn -B -DskipTests clean package
ls java-app/target/
创建一个自动化构建脚本
mkdir -p jenkins/build
vim jenkins/build/mvn.sh
#! /bin/bash
echo "***************************"
echo "Maven *********************"
echo "***************************"
docker run --rm -v $PWD/java-app:/app -v /root/.m2/:/root/.m2/ -w /app maven "$@"
测试自动化脚本
chmod u+x jenkins/build/mvn.sh
./jenkins/build/mvn.sh mvn -B -DskipTests clean package
ls java-app/target/
构建
将jar文件打包成可执行的docker
cd jenkins/build/
vim Dockerfile-java
定义一个执行jar的docker,编辑dockerfile
FROM openjdk
RUN mkdir /app
COPY *.jar /app/app.jar
CMD java -jar /app/app.jar
验证jar执行
cp ../../java-app/target/*.jar .
docker build -f Dockerfile-java -t test-java .
docker images | grep test-java
docker run test-java
# Hello 2023!
使用docker compose
vim docker-compose-java.yml
编辑docker-compose-java.yml
version: '3'
services:
app:
image: "java-project:$BUILD_TAG"
build:
context: .
dockerfile: Dockerfile-java
export BUILD_TAG=1
docker compose -f docker-compose-java.yml build
docker images | grep java
使用Jenkinsfile编写自动化流程
touch jenkins/build/build.sh
chmod u+x jenkins/build/build.sh
vim jenkins/build/build.sh
编辑build.sh
#! /bin/bash
cp -f java-app/target/*.jar jenkins/build/
echo "****************************"
echo "Building Docker Image ******"
echo "****************************"
docker-compose -f jenkins/build/docker-compose-java.yml build --no-cache
编辑Jenkinsfile
,调用build.sh
和mvn.sh
vim ~/pipeline/Jenkinsfile
pipeline {
agent any
stages {
stage('Build') {
steps {
sh '''
./jenkins/build/mvn.sh mvn -B -DskipTests clean package
./jenkins/build/build.sh
'''
}
}
}
}
测试
docker run --rm -v ./java-app:/app -v /root/.m2/:/root/.m2/ -w /app maven mvn test
ls -al java-app/target/surefire-reports/
mkdir -p jenkins/test
cd jenkins/test
cp ../build/mvn.sh .
修改Jenkinsfile
,加入test
pipeline {
agent any
stages {
stage('Build') {
steps {
sh '''
./jenkins/build/mvn.sh mvn -B -DskipTests clean package
./jenkins/build/build.sh
'''
}
}
stage('Test') {
steps {
sh './jenkins/test/mvn.sh mvn test'
}
}
}
}
推送
在云端或者VirutalBox中,创建一个新的虚拟机,模拟production生产环境
我复制的已有的虚拟机,并且重新分配了IP 10.0.0.200
在prod虚拟机上新建用户prod-user,并使用prod密钥
sudo su
hostnamectl set-hostname prod
adduser prod-user
# password 1234
usermod -aG docker prod-user
在jenkins虚拟机上生成密钥,并且分发只prod虚拟机
mkdir prod
cd prod
# empty passphrase
ssh-keygen
ssh-copy-id prod-user@10.0.0.200
ssh prod-user@10.0.0.200
创建Docker Hub仓库
- 如果没有账号,注册Docker Hub Personal account,默认有一个私有仓库的配额
- docker hub > repositories > create repository
- Name: java-project
- Visibility: Private
回到Jenkins服务器
docker login
# enter username & password
# Login Succeeded
docker images
docker tag java-project:1 s09g/java-project:1
docker push s09g/java-project:1
刷新docker hub repository页面,新的镜像上传完成
创建执行脚本
mkdir ~/pipeline/jenkins/push
cd ~/pipeline/jenkins/push
touch push.sh
chmod u+x push.sh
vim push.sh
push.sh
#! /bin/bash
echo "********************"
echo "Push ***************"
echo "********************"
echo "Logging in:"
docker login -u $ACCOUNT -p $PASSWORD
echo "Tagging image:"
docker tag $PROJECT:$BUILD_TAG $ACCOUNT/$PROJECT:$BUILD_TAG
echo "Pushing image:"
docker push $ACCOUNT/$PROJECT:$BUILD_TAG
测试
export PROJECT=java-project
export BUILD_TAG=1234
export ACCOUNT=s09g
export PASSWORD=geektime+1s!
./push.sh
修改jenkinsfile
pipeline {
agent any
stages {
stage('Build') {
steps {
sh '''
./jenkins/build/mvn.sh mvn -B -DskipTests clean package
./jenkins/build/build.sh
'''
}
}
stage('Test') {
steps {
sh './jenkins/test/mvn.sh mvn test'
}
}
stage('Push') {
steps {
sh './jenkins/push/push.sh'
}
}
}
}
部署
确定与prod远程主机之间的协议。为了简单起见,这里约定:
- 使用scp传输文本
- 传输内容包括:
- 要部署的项目名称
- 版本(BUILD_TAG)
- docker hub拉取镜像所需的账号、密码
echo $PROJECT > /tmp/.auth
echo $BUILD_TAG >> /tmp/.auth
echo $ACCOUNT >> /tmp/.auth
echo $PASSWORD >> /tmp/.auth
cat /tmp/.auth
使用scp传输/tmp/.auth文件
scp /tmp/.auth prod-user@10.0.0.200:/tmp/.auth
# ssh prod-user@10.0.0.200
# cat /tmp/.auth
# exit
创建部署脚本
mkdir ~/pipeline/jenkins/deploy
cd ~/pipeline/jenkins/deploy
touch deploy.sh
chmod u+x deploy.sh
vim deploy.sh
编辑deploy.sh
#! /bin/bash
echo $PROJECT > /tmp/.auth
echo $BUILD_TAG >> /tmp/.auth
echo $ACCOUNT >> /tmp/.auth
echo $PASSWORD >> /tmp/.auth
scp /tmp/.auth prod-user@10.0.0.200:/tmp/.auth
测试
jenkins/build/build.sh
# docker images | grep java-project
jenkins/push/push.sh
# docker hub page
jenkins/deploy/deploy.sh
# ssh prod-user@10.0.0.200
在prod机上定义docker-compose.yml模板
vim ~/docker-compose.yml
version: '3'
services:
java:
image: "$ACCOUNT/$PROJECT:$BUILD_TAG"
container_name: $PROJECT
从传输好的/tmp/.auth文本读取内容:
- 镜像
- 版本
- docker hub密码
export PROJECT=$(sed -n '1p' /tmp/.auth)
export BUILD_TAG=$(sed -n '2p' /tmp/.auth)
export ACCOUNT=$(sed -n '3p' /tmp/.auth)
export PASSWORD=$(sed -n '4p' /tmp/.auth)
echo $PROJECT
echo $BUILD_TAG
echo $ACCOUNT
echo $PASSWORD
docker login -u $ACCOUNT -p $PASSWORD
docker compose -f ~/docker-compose.yml up -d
使用脚本自动化操作,在jenkins服务器上编辑publish
touch jenkins/deploy/publish
chmod u+x jenkins/deploy/publish
vim jenkins/deploy/publish
#! /bin/bash
export PROJECT=$(sed -n '1p' /tmp/.auth)
export BUILD_TAG=$(sed -n '2p' /tmp/.auth)
export ACCOUNT=$(sed -n '3p' /tmp/.auth)
export PASSWORD=$(sed -n '4p' /tmp/.auth)
docker login -u $ACCOUNT -p $PASSWORD
docker compose -f ~/docker-compose.yml up -d
同样使用scp传输,修改deploy.sh
#!/bin/bash
echo $PROJECT > /tmp/.auth
echo $BUILD_TAG >> /tmp/.auth
echo $ACCOUNT >> /tmp/.auth
echo $PASSWORD >> /tmp/.auth
scp /tmp/.auth prod-user@10.0.0.200:/tmp/.auth
scp ./jenkins/deploy/publish prod-user@10.0.0.200:/tmp/publish
ssh prod-user@10.0.0.200 "/tmp/publish"
修改Jenkinsfile,加入自动化部署
pipeline {
agent any
stages {
stage('Build') {
steps {
sh '''
./jenkins/build/mvn.sh mvn -B -DskipTests clean package
./jenkins/build/build.sh
'''
}
}
stage('Test') {
steps {
sh './jenkins/test/mvn.sh mvn test'
}
}
stage('Push') {
steps {
sh './jenkins/push/push.sh'
}
}
stage('Deploy') {
steps {
sh './jenkins/deploy/deploy.sh'
}
}
}
}
Jenkins 实现CICD pipeline
使用Git管理源码
修改docker-compose.yml,启动gitlab
version: '3'
services:
jenkins:
container_name: jenkins
image: jenkins/docker
build:
context: pipeline
ports:
- "8080:8080"
volumes:
- $PWD/jenkins_home:/var/jenkins_home
- /var/run/docker.sock:/var/run/docker.sock
networks:
- net
remote_host:
container_name: remote-host
image: remote-host
build:
context: centos
networks:
- net
db_host:
container_name: mysql
image: mysql:5.7
environment:
- "MYSQL_ROOT_PASSWORD=1234"
volumes:
- $PWD/db_data:/var/lib/mysql
networks:
- net
web:
container_name: web
image: ansible-web
build:
context: web
ports:
- "80:80"
volumes:
- $PWD/web/html:/var/www/html
networks:
- net
git:
container_name: git-server
image: 'gitlab/gitlab-ce:latest'
hostname: 'gitlab.example.com'
ports:
- '8090:80'
volumes:
- '~/gitlab/config:/etc/gitlab'
- '~/gitlab/logs:/var/log/gitlab'
- '~/gitlab/data:/var/opt/gitlab'
networks:
- net
networks:
net:
docker compose up -d
启动,gitlab启动非常耗时
等待过程中创建gitignore
vim ~/pipeline/.gitignore
# Dockerfile
# target/
使用root:12345678登录gitlab
创建一个新的项目
- View all projects > New project > Create blank project
- name: pipeline
- group: jenkins
- Visibility Level: public
- 取消 Initialize repository with a README
- 创建
- invite memeber: ubuntu as owner
回到jenkins服务器
cd ~/pipeline
git init --initial-branch=main
git remote add origin http://ubuntu:12345678@10.0.0.215:8090/jenkins/pipeline.git
git status
git add Jenkinsfile java-app/ jenkins/
git commit -m "Initial commit"
git push -u origin main
刷新查看 http://10.0.0.215:8090/jenkins/pipeline
创建Jenkins项目
登录Jenkins,用户名密码ubuntu
- New Item > pipeline
- name:cicd-pipeline
- Advanced Project Options > Pipeline
- Definition: pipeline script from SCM
- SCM: Git
- Repository URL: http://10.0.0.215:8090/jenkins/pipeline.git
- Branches to build > */main
- Save
测试一下,Jenkins pipeline失败
sudo chown $UID:$UID /var/run/docker.sock
点击Build Now,docker路径错误
ls -l ~/jenkins_home/workspace
cd ~/jenkins_home/workspace/cicd-pipeline
pwd
# /home/ubuntu/jenkins_home/workspace/cicd-pipeline
Jenkins pipeline失败,但是会产生workspace folder
修改脚本中的执行路径
# WORKSPACE=/home/ubuntu/jenkins_home/workspace/cicd-pipeline
vim jenkins/build/mvn.sh
vim jenkins/test/mvn.sh
修改jenkins/build/build.sh
#! /bin/bash
echo "************************"
echo "Build docker image *****"
echo "************************"
cp -f java-app/target/*.jar jenkins/build/
cd jenkins/build && docker-compose -f docker-compose-java.yml build --no-cache
添加ssh key
docker cp ~/.ssh/id_rsa jenkins:/opt/prod
docker exec -ti jenkins bash
ssh -i /opt/prod prod-user@10.0.0.200
修改jenkins/deploy/deploy.sh
scp -i /opt/prod /tmp/.auth prod-user@10.0.0.200:/tmp/.auth
scp -i /opt/prod ./jenkins/deploy/publish prod-user@10.0.0.200:/tmp/publish
ssh -i /opt/prod prod-user@10.0.0.200 "/tmp/publish"
提交
git add jenkins/
git commit -m "fix path in all files"
git push -u origin main
用Jenkins传输变量
- Dashboard > Manage Jenkins > Manage Credentials > System > Global credentials (unrestricted) >
- Add credentials
- Kind: Secret text
- ID: docker-password
- Add credentials
- Kind: Secret text
- ID: docker-account
- Add credentials
- Create
修改Jenkinsfile,加入密码和其余变量
pipeline {
agent any
environment {
PASSWORD = credentials('docker-password')
PROJECT = 'java-project'
ACCOUNT = credentials('docker-account')
}
stages {
stage('Build') {
steps {
sh '''
./jenkins/build/mvn.sh mvn -B -DskipTests clean package
./jenkins/build/build.sh
'''
}
}
stage('Test') {
steps {
sh './jenkins/test/mvn.sh mvn test'
}
}
stage('Push') {
steps {
sh './jenkins/push/push.sh'
}
}
stage('Deploy') {
steps {
sh './jenkins/deploy/deploy.sh'
}
}
}
}
git add Jenkinsfile
git commit -m "add variables"
git push -u origin main
执行Jenkins pipeline
- 打开docker hub页面查看
- 登录远程prod验证
docker ps -a
docker logs java-project
添加post action
进一步完善Jenkins pipeline, 添加post action
修改Jenkinsfile
pipeline {
agent any
environment {
PASSWORD = credentials('docker-password')
PROJECT = 'java-project'
ACCOUNT = credentials('docker-account')
}
stages {
stage('Build') {
steps {
sh '''
./jenkins/build/mvn.sh mvn -B -DskipTests clean package
./jenkins/build/build.sh
'''
}
post {
success {
archiveArtifacts artifacts: 'java-app/target/*.jar', fingerprint: true
}
}
}
stage('Test') {
steps {
sh './jenkins/test/mvn.sh mvn test'
}
post {
always {
junit 'java-app/target/surefire-reports/*.xml'
}
}
}
stage('Push') {
steps {
sh './jenkins/push/push.sh'
}
}
stage('Deploy') {
steps {
sh './jenkins/deploy/deploy.sh'
}
}
}
}
提交
git add Jenkinsfile
git commit -m "post actions"
git push -u origin main
添加Git Hook
Manage Jenkins > Configure Global Security > CSRF Protection.
- 选择 Strict Crumb Issuer.
- Click on Advanced.
- Uncheck the Check the session ID box.
- Save it.
docker exec -ti git-server bash
cd /var/opt/gitlab/git-data/repositories
cd @hashed/......git
在 Main menu > Admin > Overview > Projects下选中项目,找到 Gitaly relative path
mkdir custom_hooks
touch custom_hooks/post-receive
chown git:git custom_hooks -R
vi custom_hooks/post-receive
在post-receive
中写入
#! /bin/bash
crumb=$(curl -u "ubuntu:ubuntu" -s 'http://10.0.0.215:8080/crumbIssuer/api/xml?xpath=concat(//crumbRequestField,":",//crumb)')
curl -u "ubuntu:ubuntu" -H "$crumb" -X POST http://10.0.0.215:8080/job/cicd-pipeline/build?delay=0sec
测试,将Hello 2023!
修改成Hello from pipeline!
vim java-app/src/main/java/com/mycompany/app/App.java
vim java-app/src/test/java/com/mycompany/app/AppTest.java
git add java-app
git commit -m "Hello from pipeline"
git push -u origin main
执行Jenkins pipeline
- 查看输出 / blue ocean
- 打开docker hub页面查看
- 登录远程prod验证
docker ps -a
docker logs java-project