kubesphere自定义流水线基础镜像

背景

需求：在流水线基础pod中使用python和jinja2模块来动态渲染部署文件

由于ks提供的基础镜像无法满足以上需求，在ks提供的maven镜像的基础上实现

实施

制作镜像&并推送到private image repo

FROM kubesphere/builder-maven:v3.2.0
RUN sed -i.bak -e 's|^mirrorlist=|#mirrorlist=|g' -e 's|^#baseurl=http://mirror.centos.org/centos|baseurl=https://mirrors.ustc.edu.cn/centos|g' /etc/yum.repos.d/CentOS-Base.repo && sed -e 's|^metalink=|#metalink=|g' -e 's|^#baseurl=https\?://download.fedoraproject.org/pub/epel/|baseurl=https://mirrors.ustc.edu.cn/epel/|g' -e 's|^#baseurl=https\?://download.example/pub/epel/|baseurl=https://mirrors.ustc.edu.cn/epel/|g' -i.bak /etc/yum.repos.d/epel{,-testing}.repo
RUN yum makecache && yum install python36 git-lfs -y && git lfs install && yum clean all && python3 -m pip install --upgrade pip && pip install jinja2 -i https://mirrors.ustc.edu.cn/pypi/web/simple && rm -rf /var/cache/yum

在ks中添加自定义镜像

refer
https://kubesphere.io/zh/docs/v3.3/devops-user-guide/how-to-use/pipelines/customize-jenkins-agent/

注意点是要在最下面的jenkins_user.yaml中添加自定义的基础镜像，直接复制粘贴更改name和image和label字段即可，注意containers的name字段。

          - name: "maven-jinja2"
            namespace: "kubesphere-devops-worker"
            label: "maven-jinja2"
            nodeUsageMode: "EXCLUSIVE"
            idleMinutes: 0
            imagePullSecrets:
            - name: xxx
            containers:
            - name: "maven"
              image: "xxx/kubesphere-builder-maven:xxx"
              command: "cat"
              args: ""
              ttyEnabled: true
              privileged: false
              resourceRequestCpu: "100m"
              resourceLimitCpu: "4000m"
              resourceRequestMemory: "100Mi"
              resourceLimitMemory: "8192Mi"
            - name: "jnlp"
              image: "jenkins/inbound-agent:4.10-2"
              args: "^${computer.jnlpmac} ^${computer.name}"
              resourceRequestCpu: "50m"
              resourceLimitCpu: "500m"
              resourceRequestMemory: "400Mi"
              resourceLimitMemory: "1536Mi"
            workspaceVolume:
              emptyDirWorkspaceVolume:
                memory: false
            volumes:
            - hostPathVolume:
                hostPath: "/var/run/docker.sock"
                mountPath: "/var/run/docker.sock"
            - hostPathVolume:
                hostPath: "/var/data/jenkins_maven_cache"
                mountPath: "/root/.m2"
            - hostPathVolume:
                hostPath: "/var/data/jenkins_sonar_cache"
                mountPath: "/root/.sonar/cache"
            yaml: |
              spec:
                affinity:
                  nodeAffinity:
                    preferredDuringSchedulingIgnoredDuringExecution:
                    - weight: 1
                      preference:
                        matchExpressions:
                        - key: node-role.kubernetes.io/worker
                          operator: In
                          values:
                          - ci
                tolerations:
                - key: "node.kubernetes.io/ci"
                  operator: "Exists"
                  effect: "NoSchedule"
                - key: "node.kubernetes.io/ci"
                  operator: "Exists"
                  effect: "PreferNoSchedule"
                containers:
                - name: "maven"
                  resources:
                    requests:
                      ephemeral-storage: "1Gi"
                    limits:
                      ephemeral-storage: "10Gi"
                  volumeMounts:
                  - name: config-volume
                    mountPath: /opt/apache-maven-3.5.3/conf/settings.xml
                    subPath: settings.xml
                volumes:
                  - name: config-volume
                    configMap:
                      name: ks-devops-agent
                      items:
                      - key: MavenSetting
                        path: settings.xml
                securityContext:
                  fsGroup: 1000

在流水线中使用自定义镜像做为slave

这点有个小技巧，可以通过ks的图形化界面那里选择node的时候，有个下拉框里面看看是否有自定义的label名字

@Library('xxx') _

pipeline {
  environment {
  }

  agent {
    node {
      label 'maven-jinja2' # 指定自定义的label名字
    }
  }

  stages {
    stage('CI (Git checkout & Build image)') { 
      agent {
        node {
          label 'maven-jinja2'
        }
      }

      steps {
        container('maven') { # 注意container
          script {
          }

          script {
          }

          script {
          }

          script {
          }

        }
      }
    }

  }
}

报错
java.net.ProtocolException: Expected HTTP 101 response but was '400 Bad Request

refer

使用自定义基础镜像之后报错4xx
https://ask.kubesphere.io/forum/d/7769-devops-expected-http-101-response-but-was-400-bad-request