Kubernetes下Jenkins安装及配置

角色授权

apiVersion: v1
kind: ServiceAccount
metadata:
  name: jenkins-sa
  namespace: kube-ops

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: jenkins-cr
rules:
  - apiGroups: ["extensions", "apps"]
    resources: ["deployments"]
    verbs: ["create", "delete", "get", "list", "watch", "patch", "update"]
  - apiGroups: [""]
    resources: ["services"]
    verbs: ["create", "delete", "get", "list", "watch", "patch", "update"]
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["create","delete","get","list","patch","update","watch"]
  - apiGroups: [""]
    resources: ["pods/exec"]
    verbs: ["create","delete","get","list","patch","update","watch"]
  - apiGroups: [""]
    resources: ["pods/log"]
    verbs: ["get","list","watch"]
  - apiGroups: [""]
    resources: ["secrets"]
    verbs: ["get"]

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: jenkins-crd
roleRef:
  kind: ClusterRole
  name: jenkins-cr
  apiGroup: rbac.authorization.k8s.io
subjects:
- kind: ServiceAccount
  name: jenkins-sa
  namespace: kube-ops

二、创建持久化存储卷，以下为参考

apiVersion: v1
kind: PersistentVolume
metadata:
  labels:
    alicloud-pvname: jenkins-database-pv
  name: jenkins-database-pv
spec:
  accessModes:
    - ReadWriteMany
  capacity:
    storage: 200Gi
  claimRef:
    apiVersion: v1
    kind: PersistentVolumeClaim
    name: jenkins-database-pvc
    namespace: kube-ops
  flexVolume:
    driver: alicloud/nas
    options:
      modeType: non-recursive
      path: /jenkins-database
      server: 1f62f4b505-gvo97.cn-beijing.nas.aliyuncs.com
      vers: '3'
  persistentVolumeReclaimPolicy: Retain
  storageClassName: nas
  volumeMode: Filesystem
  
  ---
  apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: jenkins-database-pvc
  namespace: kube-ops
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 200Gi
  selector:
    matchLabels:
      alicloud-pvname: jenkins-database-pv
  storageClassName: nas
  volumeMode: Filesystem
  volumeName: jenkins-database-pv

部署Jenkins

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: jenkins
  name: jenkins
  namespace: kube-ops
spec:
  selector:
    matchLabels:
      app: jenkins
  template:
    metadata:
      labels:
        app: jenkins
    spec:
      containers:
        - env:
            - name: JAVA_OPTS
              value: >-
                -XshowSettings:vm -Dhudson.slaves.NodeProvisioner.initialDelay=0
                -Dhudson.slaves.NodeProvisioner.MARGIN=50
                -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85
                -Duser.timezone=Asia/Shanghai
          image: 'registry.cn-beijing.aliyuncs.com/ienglish/jenkins:lts'
          imagePullPolicy: IfNotPresent
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /login
              port: 8080
              scheme: HTTP
            initialDelaySeconds: 60
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
          name: jenkins
          ports:
            - containerPort: 8080
              name: web
              protocol: TCP
            - containerPort: 50000
              name: agent
              protocol: TCP
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /login
              port: 8080
              scheme: HTTP
            initialDelaySeconds: 60
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
          resources:
            limits:
              cpu: '2'
              ephemeral-storage: 8Gi
            requests:
              cpu: '2'
              ephemeral-storage: 8Gi
          volumeMounts:
            - mountPath: /var/jenkins_home
              name: volume-jenkins-database-pvc
      dnsPolicy: ClusterFirst
      imagePullSecrets:
        - name: registry.cn-beijing.aliyuncs.com
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      serviceAccount: jenkins-sa
      serviceAccountName: jenkins-sa
      volumes:
        - name: volume-jenkins-database-pvc
          persistentVolumeClaim:
            claimName: jenkins-database-pvc
 
---
apiVersion: v1
kind: Service
metadata:
  name: jenkins-svc
  namespace: kube-ops
spec:
  clusterIP: 172.21.10.174
  ports:
    - name: web
      port: 8080
      protocol: TCP
      targetPort: 8080
    - name: agent
      port: 50000
      protocol: TCP
      targetPort: 50000
  selector:
    app: jenkins
  sessionAffinity: None
  type: ClusterIP

登录Jenkins

如果Jenkins启动报以下错误，对挂载目录chown -R 1000 授权即可

[root@master manifests]# kubectl logs jenkins-688c6cd5fd-lj6zg -n devops 
touch: cannot touch '/var/jenkins_home/copy_reference_file.log': Permission denied
Can not write to /var/jenkins_home/copy_reference_file.log. Wrong volume permissions?

登录Jenkins后，安装kubernetes插件，然后点击配置管理 --> 系统配置 --> 新增cloud

** Jenkins 通道必须得配置，如未配置，将会导致Jenkins-slave启动后无法通信得问题 **

** 节点选择器配置后，在K8S指定节点配置相应标签 **

使用小技巧

安装插件build user vars plugin，在jenkinsfile中输入一下代码：

stage('Initialization') {
                buildName "${ENV}--${PROJECT}--${BUILD_NUMBER}"
            }

效果如下：