外部Jenkins连接kubernetes部署deployment

应用版本信息

应用	版本
Jenkins	2.358
kubenetest	v1.22.2

最终效果图

配置Jenkins机器通过kubectl连接kubernetes

我这里是打算直接在 Jenkins 的机器上安装 kubectl 来访问 kubernetes 集群
但是生产环境咱们需要考虑到安全问题，该做法还是存在一定的问题
读者按自己实际情况去构思即可

]# cd /root/.kube
]# scp config ${jenkins}:/root/.kube  
# 这里是把 k8s 的 config 文件拷贝到 Jenkins 机器上的 /root/.kube/ 目录下，如果没有则自行创建

# 去到 Jenkins 机器上安装 kubeclt
]# yum -y install kubectl
# 然后即可查看到集群信息
]# kubectl get nodes
NAME    STATUS   ROLES                  AGE    VERSION
k8s01   Ready    control-plane,master   301d   v1.22.2
k8s02   Ready    <none>                 300d   v1.22.2
k8s03   Ready    <none>                 300d   v1.22.2
k8s04   Ready    <none>                 300d   v1.22.2
k8s05   Ready    <none>                 300d   v1.22.2

脚本说明

deploy_front_base.groovy

以下都是基于Jenkins share library，有兴趣的可自行查看

import org.devops.otherTools

def call(Map runMap) {

    def timeStr=new Date().format('yyyyMMddHHmm')
    def mytools = new org.devops.otherTools()

    pipeline {
        agent {
            label runMap.RUN_NODE
        }
        environment {
            serverName = "${runMap.SERVERNAME}"
            gitBranch = "${runMap.BRANCH}"
            gitUserId = "${runMap.USERID}"
            gitUrl = "${runMap.URL}"
            pbBranch = "${runMap.PLAYBOOKBRANCH}"
            pbUrl = "${runMap.PLAYBOOKURL}"
            dockerfileName = "${runMap.DOCKERFILE}"
            harborName = "${runMap.HARBORNAME}"
            harborProject = "${runMap.HARBORPROJECT}"
            imageName = "${harborName}/${harborProject}/${serverName}:${timeStr}"
            k8sNamespace = "${runMap.K8SNS}"
            k8sYaml = "${runMap.K8SYAML}"
            k8sPort = "${runMap.K8SPORT}"
            k8sNodePort = "${runMap.K8SNODEPORTb}"
            testPk = "test-pk"
        }

        options {
            // 表示保留10次构建历史
            buildDiscarder(logRotator(numToKeepStr: '10'))\
            // 输出构建的时间信息
            timestamps()
        }

        stages {
            stage('拉取剧本') {
                steps {
                    checkout([$class: 'GitSCM', 
                    branches: [[name: "${pbBranch}"]], 
                    extensions: [], 
                    userRemoteConfigs: [[credentialsId: "${gitUserId}", url: "${pbUrl}"]]])
                }
            }

            stage('rename dir') {
                steps {
                    sh """
                        mkdir -p ${WORKSPACE}/../${testPk} && cp -rf ${WORKSPACE}/* ${WORKSPACE}/../${testPk}
                    """
                }
            }

            stage('拉取代码') {
                steps {
                    script {
                        mytools.printMsg('hello world')
                    }
                    checkout([$class: 'GitSCM', 
                    branches: [[name: "${gitBranch}"]], 
                    extensions: [], 
                    userRemoteConfigs: [[credentialsId: "${gitUserId}", url: "${gitUrl}"]]])
                }
            }

            stage('制作容器镜像') {
                steps {
                    sh """
                        export serverName=${serverName}
                        envsubst < ${WORKSPACE}/../${testPk}/${dockerfileName} | docker build  -t ${imageName} . -f -
                    """
                }
            }

            stage('上传镜像') {
                steps {
                    withCredentials([usernamePassword(credentialsId: '5c4562c3-893e-4300-a8f7-3f91c7521e75', passwordVariable: 'harborPass', usernameVariable: 'harborUser')]) {
                        sh """
                            docker login -u ${harborUser} -p ${harborPass} ${harborName}
                            docker push ${imageName}
                        """
                    }
                }
            }

            stage('部署到k8s') {
                steps {
                    sh """
                        export serverName=${serverName}
                        export k8sNamespace=${k8sNamespace}
                        export imageName=${imageName}
                        envsubst < ${WORKSPACE}/../${testPk}/${k8sYaml} | kubectl apply -f -
                    """
                }
            }
        }
    }
}

Dockerfile

此处使用的是多重构建，配合 envsubst 实现动态参数构建镜像
使用方式如下：

envsubst < Dockerfile | docker build  -t ${imageName} . -f -

# build stage
FROM golang:1.18-alpine AS builder

ENV CGO_ENABLED 0
ENV GOOS linux
ENV GOPROXY https://goproxy.cn,direct

WORKDIR /app

COPY ./* /app/

RUN go build -o ${serverName} .

# run stage
FROM alpine:3.16.0

LABEL maintainer="Sky"

WORKDIR /

COPY --from=builder /app/${serverName} /

RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && mkdir /lib64 && ln -s /lib/libc.musl-x86_64.so.1 /lib64/ld-linux-x86-64.so.2

EXPOSE 8000

CMD ["./${serverName}"]

Deployment.yaml

使用方式如下：

envsubst < Deployment.yaml | kubectl apply -f -

kind: Service
apiVersion: v1
metadata:
  name: ${serverName}-svc
  namespace: ${k8sNamespace}
spec:
  type: NodePort
  ports:
    - port: 8000
      targetPort: 8000
      nodePort: 30800
  selector:
    app: ${serverName}
    
---

apiVersion: apps/v1
kind: Deployment  
metadata:
  name:  ${serverName}
  namespace: ${k8sNamespace}
spec:
  replicas: 3
  selector:
    matchLabels:
      app: ${serverName}
  template:
    metadata:
      labels:
        app: ${serverName}
    spec:
      containers:
      - name: ${serverName}
        image: ${imageName}
        ports:
        - containerPort: 8000

Map.groovy

该脚本定义好 Map
然后传递并调用写好的 pipeline

@Library('jenkins_lib@main') _

def runMap = [:]
    // 定义项目构建运行的 NODE
    runMap.put('RUN_NODE','master')

    // 项目名
    runMap.put('SERVERNAME', 'hellogo')

    // git 分支
    runMap.put('BRANCH', '*/main')
    runMap.put('PLAYBOOKBRANCH', '*/main')

    // git userId
    runMap.put('USERID', 'b6dc9f5f-581b-1234-82d5-55c3ebc39b32')

    // git url
    runMap.put('URL', 'http://10.10.3.21/golang/test-build.git')
    runMap.put('PLAYBOOKURL', 'http://10.10.3.21/testall/playbook.git')
    
    // dockerfile config
    runMap.put('DOCKERFILE', 'helloGoDockerfile')

    // harbor config
    runMap.put('HARBORNAME', 'my.harbor.com')
    runMap.put('HARBORPROJECT', 'test')

    // kubernetes config
    runMap.put('K8SNS', 'test')
    runMap.put('K8SYAML', 'helloGoDeployment.yaml')
    runMap.put('K8SPORT', '8000')
    runMap.put('K8SNODEPORTb', '30800')
    
// 此处是真正运行上面 pipeline 脚本的地方，并把 Map 传递进去
deploy_front_base(runMap)