Jenkins打包springboot项目到k8s

一、Jenkins配置

1.1、pipeline插件安装

二、拉取git代码

pipeline {
    agent any
	stages {
		stage('Checkout') {
			steps {
				echo "1.克隆git仓库代码"
				git branch: 'dev', credentialsId: 'c253ee4c-f900-422b-b3f2-b83090ae6a9b', url: 'https://gitee.com/luckywxn/WeaveErpSoa.git'
				script {
					build_tag = sh(returnStdout: true, script: 'git rev-parse --short HEAD').trim()
				  }
				echo "${build_tag}"
			}
		}
	}
}

三、Maven打包spring项目

pipeline {
    agent any
	stages {
		stage('Checkout') {
			steps {
				echo "1.克隆git仓库代码"
				git branch: 'dev', credentialsId: 'c253ee4c-f900-422b-b3f2-b83090ae6a9b', url: 'https://gitee.com/luckywxn/WeaveErpSoa.git'
				script {
					build_tag = sh(returnStdout: true, script: 'git rev-parse --short HEAD').trim()
				  }
				echo "${build_tag}"
			}
		}
		stage('Maven Build') {
            steps {
				echo "2.Maven 打包项目"
                sh "mvn --version"
				sh 'pwd'
				sh 'mvn clean package -DskipTests'
				echo '构建完成'
            }
        }
	}
}

三、docker构建镜像

pipeline {
    agent any
	stages {
		stage('Checkout') {
			steps {
				echo "1.克隆git仓库代码"
				git branch: 'dev', credentialsId: 'c253ee4c-f900-422b-b3f2-b83090ae6a9b', url: 'https://gitee.com/luckywxn/WeaveErpSoa.git'
				script {
					build_tag = sh(returnStdout: true, script: 'git rev-parse --short HEAD').trim()
				  }
				echo "${build_tag}"
			}
		}
		stage('Maven Build') {
            steps {
				echo "2.Maven 打包项目"
                sh "mvn --version"
				sh 'pwd'
				sh 'mvn clean package -DskipTests'
				echo '构建完成'
            }
        }
		stage('Docker Build') {
            steps {
				echo "3.Docker 构造镜像"
				sh "docker -v"
				sh 'pwd'
				sh """cd /var/jenkins_home/workspace/test/weaveserviceservice/src/main/docker/ 
				ls
				docker -H tcp://127.0.0.1:80 build -t springbootapp:${build_tag} ."""
            }
        }
	}
}

遇到的问题：
在使用docker build命名的时候，报如下错误

Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

解决方案：
在jenkins 启动的docker-compose文件中增加如下配置

/var/run/docker.sock:/var/run/docker.sock

四、docker推送镜像

def servicename = "weaveservice"
node {
	stage('Checkout') {
		echo "1.克隆git仓库代码"
		git branch: 'dev', credentialsId: 'c253ee4c-f900-422b-b3f2-b83090ae6a9b', url: 'https://gitee.com/luckywxn/WeaveErpSoa.git'
		script {
			build_tag = sh(returnStdout: true, script: 'git rev-parse --short HEAD').trim()
		  }
		echo "${build_tag}"
	}
	stage('Maven Build') {
		echo "2.Maven 打包项目"
		sh "mvn --version"
		sh 'pwd'
		sh 'mvn clean package -DskipTests'
		echo '构建完成'
	}
	stage('Docker Build') {
		echo "3.Docker 构造镜像"
		sh "docker -v"
		sh 'pwd'
		sh """cd /var/jenkins_home/workspace/test/weaveserviceservice/src/main/docker/ 
		ls
		docker build -t 127.0.0.1:80/dev/${servicename}:${build_tag} ."""
	}
	stage('Docker Push') {
		echo "4.Docker 推送镜像"
		withCredentials([usernamePassword(credentialsId: "0fbd81ff-a45b-4e8b-9355-a2a5a36438c0", passwordVariable: "password", usernameVariable: "username")]) {
			sh "docker login 127.0.0.1:80 -u ${username} -p ${password}"
			sh "docker push 127.0.0.1:80/dev/${servicename}:${build_tag}"
		}
	}
}

遇到的问题：
在使用docker build命令的时候，报如下错误

Cannot perform an interactive login from a non TTY device 

解决方案：
参考博文Docker、Jenkins、Harbor 构建镜像部署 SpringBoot 项目，先不在jenkins中执行，先在宿主机上执行是否可以成功，然后在jenkins中编写对应脚本就可以了。

五、kubelctl执行

def harborurl = "127.0.0.1:80"
def servicename = "weaveservice"
node {
	stage('Checkout') {
		echo "1.克隆git仓库代码"
		git branch: 'dev', credentialsId: 'c253ee4c-f900-422b-b3f2-b83090ae6a9b', url: 'https://gitee.com/luckywxn/WeaveErpSoa.git'
		script {
			build_tag = sh(returnStdout: true, script: 'git rev-parse --short HEAD').trim()
		  }
		echo "${build_tag}"
	}
	stage('Maven Build') {
		echo "2.Maven 打包项目"
		sh "mvn --version"
		sh 'pwd'
		sh 'mvn clean package -DskipTests'
		echo '构建完成'
	}
	stage('Docker Build') {
		echo "3.Docker 构造镜像"
		sh "docker -v"
		sh 'pwd'
		sh """cd /var/jenkins_home/workspace/test/weaveserviceservice/src/main/docker/ 
		ls
		docker build -t ${harborurl}/dev/${servicename}:${build_tag} ."""
	}
	stage('Docker Push') {
		echo "4.Docker 推送镜像"
		withCredentials([usernamePassword(credentialsId: "0fbd81ff-a45b-4e8b-9355-a2a5a36438c0", passwordVariable: "password", usernameVariable: "username")]) {
			sh "docker login ${harborurl} -u ${username} -p ${password}"
			sh "docker push ${harborurl}/dev/${servicename}:${build_tag}"
		}
	}
	stage('Deploy') {
		echo "5. Deploy Kubernetes"
		sh "kubectl version"
		sh """cd /var/jenkins_home/workspace/test/weaveserviceservice/ 
		ls
		sed -i 's/<PROJECT_NAMESPACE>/app/' k8s.yaml
		sed -i 's/<DOCKER_REGISTRY>/${harborurl}/' k8s.yaml
		sed -i 's/<PROJECT_NAME>/${servicename}/' k8s.yaml
		sed -i 's/<BUILD_TAG>/${build_tag}/' k8s.yaml
		kubectl delete -f .
		kubectl apply -f k8s.yaml --record
		"""
	}
}

遇到的问题一：

在使用kubectl version命令的时候，报如下错误

Error from server (Forbidden): <html><head><meta http-equiv='refresh' content='1;url=/login?from=%2Fswagger-2.0.0.pb-v1%3Ftimeout%3D32s'/><script>window.location.replace('/login?from=%2Fswagger-2.0.0.pb-v1%3Ftimeout%3D32s');</script></head><body style='background-color:white; color:white;'>


Authentication required
<!--
-->

</body></html>

解决方案：
参考博文从 jenkins 容器运行 kubectl 命令，jenkins配置的环境变量中设置kubectl 的认证信息

遇到的问题二：

在使用kubectl apply -f k8s.yaml --record命令的拉取镜像的时候，报如下错误

container "weaveservice" in pod "weaveservice-dad798uvdkkd"is waiting pulled 

解决方案：
参考博文k8s 1.23 拉取 harbor镜像失败，配置好后还是报错，后面突然发现，pod是在node节点拉取的镜像，然后在node节点的宿主机执行docker login 命令，发现又有如下报错：

Error response from daemon: Get "https://192.168.1.7:80/v2/": http: server gave HTTP response to HTTPS client

然后参考博文 Docker login 登录私服，报错； http: server gave HTTP response to HTTPS client操作后，docker login 命令执行成功，镜像拉取下来之后，pod也就自动启动了。大功告成！！！

参考资料

1. jenkins流水线部署springboot应用到k8s集群（k3s+jenkins+gitee+maven+docker）（2）
2. jenkins发布springboot到k8s
3. Jenkins中无法启动docker daemon的解决和思考
4. SpringBoot 应用如何部署到 K8S 上？
5. jenkins 上的 docker 登录
6. Docker、Jenkins、Harbor 构建镜像部署 SpringBoot 项目
7. k8s 1.23 拉取 harbor镜像失败
8. K8S部署应用时从harbor拉取镜像失败
9. Docker login 登录私服，报错； http: server gave HTTP response to HTTPS client