实践005-Gitlab CICD全项目整合

木二_

已于 2025-05-08 23:47:49 修改

阅读量1.1k

点赞数 30

分类专栏：运维工具文章标签： gitlab ci/cd

于 2025-05-08 15:36:45 首次发布

本文链接：https://blog.csdn.net/tagaochen1276/article/details/147793493

版权

运维工具专栏收录该内容

12 篇文章

订阅专栏

环境准备

单独测试前后端项目的时候已成功部署相关应用，为便于后续整合，将已部署的应用进行清理。

root@master01:~# kubectl -n gitlabci delete deployments.apps deploy-apiserver-ci deploy-webui-ci
root@master01:~# kubectl -n gitlabci delete service service-apiserver-ci service-webui-ci
root@master01:~# kubectl delete ns gitlabci

root@master01:~# kubectl get ns

整个环境需要发布镜像至阿里云，需要将应用部署到 Kubernetes 。
因此提前在 gitlab 中创建 ALIYUN_USER 和 ALIYUN_PASSWORD 、KUBE_CONFIG 变量，配置阿里云镜像推送的账号和密码。

集成Kubernetes

当前 Gitlab 的 runner 是基于 helm 部署 gitla 的同时配套部署的，即 runner 是运行在 Kubernetes 中的一个 Pod，runner 类型是 Kubernetes ，如下所示：

root@master01:~# kubectl -n gitlab exec -ti mygitlab-gitlab-runner-798986f578-h2thf -- bash
camygitlab-gitlab-runner-798986f578-h2thf:/$ cat /home/gitlab-runner/.gitlab-runner/config.toml
#……
[[runners]]
#……
  executor = "kubernetes"

因此该 runner 后续需要直接在 Kubernetes 中部署业务，需要安装 kubectl 命令，以及配置 kubeconfig 上下文。

从而需要提前将 kubeconfig 内容以变量形式引入到 runner Pod 中。

root@master01:~# echo $(cat ~/.kube/config | base64) | tr -d " "
YXBpVmVyc2lvbjogdjEKY2x1c3RlcnM6Ci0gY2x1c3RlcjoKICA……

添加变量 KUBE_CONFIG 。
227

提示：由于后续流水线中作业有 main 和 tag 两种触发方式，因此建议将变量取消受保护。
同时对于 Kubernetes 的部署可参考：附042.Kubernetes_v1.33.0生成环境高可用部署方案

Gitlab CICD项目整合

项目整合

整合设计

当前后端 java 和前端 web 的构建、测试、编译基于学习目的，都由独立项目通过 gitlab 验证，基于生产环境需要，现需要整合至一个项目中，从而将多个子项目放在一个项目代码仓中。

gitlab 创建总项目： mycicd 。
由于整合后的新项目为 mycicd ，因此原有部分文件的内容涉及路径部分均需要稍作调整，sonarqube代码检查对接的gitlab项目也需要重新调整。
213

[root@gitclient ~]# git clone git@gitlab.linuxsb.com:mygroup/mycicd.git
[root@gitclient ~]# cd mycicd/

复制前后端项目至该目录。

[root@gitclient mycicd]# cp -rp ../apiserver .
[root@gitclient mycicd]# cp -rp ../webui .
[root@gitclient mycicd]# tree -L 2 .
.
├── apiserver
│   ├── deployjavaci.yaml
│   ├── deployjavaprod.yaml
│   ├── deployjavatest.yaml
│   ├── Dockerfile
│   ├── HELP.md
│   ├── mvnw
│   ├── mvnw.cmd
│   ├── pom.xml
│   ├── README.md
│   └── src
├── README.md
└── webui
    ├── babel.config.js
    ├── deploywebuici.yaml
    ├── deploywebuiprod.yaml
    ├── deploywebuitest.yaml
    ├── Dockerfile
    ├── jsconfig.json
    ├── node_modules
    ├── package.json
    ├── package-lock.json
    ├── public
    ├── README.md
    ├── src
    └── vue.config.js

后端Java项目部署

后端Java项目静态检查

参考实践002-Gitlab CICD静态代码检查，重新配置 sonarqube 对 gitlab 新建的 mycicd 项目的检查。

214

后端Java项目镜像构建

目录有所调整。

[root@gitclient mycicd]# vim apiserver/Dockerfile
FROM uhub.service.ucloud.cn/imxhy/maven:3.8.5-openjdk-17

MAINTAINER xhy@itzgr.cn

RUN cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && echo "Asia/Shanghai" > /etc/timezone

EXPOSE 8080

WORKDIR /opt/apiservice

COPY target/apiservice-0.0.1-SNAPSHOT.jar ./

ENTRYPOINT ["java","-jar","/opt/apiservice/apiservice-0.0.1-SNAPSHOT.jar"]

创建Java项目部署文件

调整如下 ci 、 test 、 prod 部署文件。

ci 部署文件

[root@gitclient mycicd]# vim apiserver/deployjavaci.yaml
---
apiVersion: v1
kind: Namespace
metadata:
  name: gitlabci

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: deploy-apiserver-ci
  namespace: gitlabci
spec:
  replicas: 2
  revisionHistoryLimit: 5
  selector:
    matchLabels:
      app: apiserver-ci
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 25%
      maxSurge: 25%
  template:
    metadata:
      labels:
        app: apiserver-ci
    spec:
      containers:
        - name: apiserver-ci
          env:
            - name: TZ
              value: Asia/Shanghai
          image: __POD_CONTAINERS_IMAGE__
          imagePullPolicy: IfNotPresent
          ports:
          - containerPort: 8080
            protocol: TCP
          readinessProbe:
            httpGet:
              path: /demo/hello
              port: 8080
              scheme: HTTP
            initialDelaySeconds: 30
            periodSeconds: 10
          livenessProbe:
            httpGet:
              path: /demo/hello
              port: 8080
              scheme: HTTP
            initialDelaySeconds: 30
            periodSeconds: 10

---
apiVersion: v1
kind: Service
metadata:
  name: service-apiserver-ci
  namespace: gitlabci
spec:
  ports:
  - nodePort: 32101
    port: 8080
    protocol: TCP
    targetPort: 8080
  selector:
    app: apiserver-ci
  sessionAffinity: ClientIP
  type: NodePort

test 部署文件

[root@gitclient mycicd]# vim apiserver/deployjavatest.yaml
---
apiVersion: v1
kind: Namespace
metadata:
  name: gitlabtest

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: deploy-apiserver-test
  namespace: gitlabtest
spec:
  replicas: 2
  revisionHistoryLimit: 5
  selector:
    matchLabels:
      app: apiserver-test
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 25%
      maxSurge: 25%
  template:
    metadata:
      labels:
        app: apiserver-test
    spec:
      containers:
        - name: apiserver-test
          env:
            - name: TZ
              value: Asia/Shanghai
          image: __POD_CONTAINERS_IMAGE__
          imagePullPolicy: IfNotPresent
          ports:
          - containerPort: 8080
            protocol: TCP
          readinessProbe:
            httpGet:
              path: /demo/hello
              port: 8080
              scheme: HTTP
            initialDelaySeconds: 30
            periodSeconds: 10
          livenessProbe:
            httpGet:
              path: /demo/hello
              port: 8080
              scheme: HTTP
            initialDelaySeconds: 30
            periodSeconds: 10

---
apiVersion: v1
kind: Service
metadata:
  name: service-apiserver-test
  namespace: gitlabtest
spec:
  ports:
  - nodePort: 32102
    port: 8080
    protocol: TCP
    targetPort: 8080
  selector:
    app: apiserver-test
  sessionAffinity: ClientIP
  type: NodePort

prod 部署文件

[root@gitclient mycicd]# vim apiserver/deployjavaprod.yaml
---
apiVersion: v1
kind: Namespace
metadata:
  name: gitlabprod

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: deploy-apiserver-prod
  namespace: gitlabprod
spec:
  replicas: 2
  revisionHistoryLimit: 5
  selector:
    matchLabels:
      app: apiserver-prod
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 25%
      maxSurge: 25%
  template:
    metadata:
      labels:
        app: apiserver-prod
    spec:
      containers:
        - name: apiserver-prod
          env:
            - name: TZ
              value: Asia/Shanghai
          image: __POD_CONTAINERS_IMAGE__
          imagePullPolicy: IfNotPresent
          ports:
          - containerPort: 8080
            protocol: TCP
          readinessProbe:
            httpGet:
              path: /demo/hello
              port: 8080
              scheme: HTTP
            initialDelaySeconds: 30
            periodSeconds: 10
          livenessProbe:
            httpGet:
              path: /demo/hello
              port: 8080
              scheme: HTTP
            initialDelaySeconds: 30
            periodSeconds: 10

---
apiVersion: v1
kind: Service
metadata:
  name: service-apiserver-prod
  namespace: gitlabprod
spec:
  ports:
  - nodePort: 32103
    port: 8080
    protocol: TCP
    targetPort: 8080
  selector:
    app: apiserver-prod
  sessionAffinity: ClientIP
  type: NodePort

创建完整流水线

创建如下流水线，并且将后端 Java 项目的编译构建阶段全部整合到一起。并且最后直接使用 git clone 使用 autotest 项目进行最后的测试。

即对于后端 Java 项目，整合 UnitTest+compile+sonarqube-check ----> build ----> deploy_java_ci/deploy_java_test ----> check_java_ci_pod/check_java_test_pod ----> test 全链路流程。

[root@gitclient mycicd]# vim apiserver/.gitlab-ci.yml
stages:
  - compile
  - build
  - deploy
  - check
  - test

variables:
  KUBECONFIG: "/.kube/config"
  PROJECT_DIR: "${CI_PROJECT_DIR}/apiserver"
  GITLAB_HOST: gitlab.linuxsb.com
  GITLAB_PORT: "32222"
  REPO_URL: git@${GITLAB_HOST}:mygroup/autotest.git

unittest_java:
  stage: compile
  image: maven:3.8.5-openjdk-17
  script:
    - cd ${PROJECT_DIR}
    - mvn verify -Dmaven.test.failure.ignore=true
    - ls target/surefire-reports/*.xml
  rules:
    - if: $CI_COMMIT_BRANCH == "main" || $CI_COMMIT_TAG
  artifacts:
    when: always
    reports:
      junit:
        - apiserver/target/surefire-reports/TEST-*.xml
        - apiserver/target/failsafe-reports/TEST-*.xml
  tags:
    - study-runner

compile_java:
  stage: compile
  image: uhub.service.ucloud.cn/imxhy/maven:3.8.5-openjdk-17
  artifacts:
    paths:
      - apiserver/target/apiservice-0.0.1-SNAPSHOT.jar
  script:
    - pwd
    - cd ${PROJECT_DIR}
    - mvn clean
    - mvn compile
    - mvn package -Dmaven.test.skip=true
    - ls target
  rules:
    - if: $CI_COMMIT_BRANCH == "main" || $CI_COMMIT_TAG
  tags:
    - study-runner

sonarqube_check_java:
  stage: compile
  image: maven:3.8.5-openjdk-17
  variables:
    SONAR_USER_HOME: "${CI_PROJECT_DIR}/apiserver/.sonar"
    GIT_DEPTH: "0"
  cache:
    key: "${CI_JOB_NAME}"
    paths:
      - .sonar/cache
  script:
    - cd ${PROJECT_DIR}
    - mvn verify sonar:sonar -Dsonar.projectKey=mygroup_mycicd_AZaRvvQBjzPXArMYpIcZ
  allow_failure: true
  rules:
    - if: $CI_COMMIT_BRANCH == "main"
  tags:
    - study-runner

build_java:
  stage: build
  image: uhub.service.ucloud.cn/imxhy/executor:v1.9.0-debug
  needs:
    - unittest_java
    - compile_java
  script:
    - cd ${PROJECT_DIR}
    - ls target
    - IMAGE_TAG=$(echo "${CI_COMMIT_TIMESTAMP}" | sed 's/T/_/g; s/-//g; s/://g' | cut -c1-15)
    - IMAGE_TAG_TO_INSTALL=${CI_COMMIT_TAG:-$IMAGE_TAG}
    - echo ${IMAGE_TAG_TO_INSTALL}
    - mkdir -p /kaniko/.docker
    - echo "{\"auths\":{\"registry.cn-hangzhou.aliyuncs.com\":{\"username\":\"${ALIYUN_USER}\",\"password\":\"${ALIYUN_PASSWORD}\"}}}" > /kaniko/.docker/config.json
    - cat /kaniko/.docker/config.json
    - echo ${PROJECT_DIR}
    - echo {\"username\":\"${ALIYUN_USER}\",\"password\":\"${ALIYUN_PASSWORD}\"}
    - ls ${PROJECT_DIR}/Dockerfile
    - >
      /kaniko/executor
      --context "${PROJECT_DIR}"
      --dockerfile "${PROJECT_DIR}/Dockerfile"
      --destination "registry.cn-hangzhou.aliyuncs.com/xhyimages/apiservice:${IMAGE_TAG_TO_INSTALL}"
      --registry-mirror "https://dbzucv6w.mirror.aliyuncs.com"
  rules:
    - if: $CI_COMMIT_BRANCH == "main" || $CI_COMMIT_TAG
  tags:
    - study-runner

deploy_java_ci:
  stage: deploy
  image: uhub.service.ucloud.cn/imxhy/kubectl:1.33.0
  script:
    - mkdir -p /.kube
    - echo $KUBE_CONFIG | base64 -d > $KUBECONFIG
    - kubectl version
    - IMAGE_TAG=$(echo "${CI_COMMIT_TIMESTAMP}" | sed 's/T/_/g; s/-//g; s/://g' | cut -c1-15)
    - IMAGE_TAG_TO_INSTALL=${CI_COMMIT_TAG:-$IMAGE_TAG}
    - echo $IMAGE_TAG_TO_INSTAL
    - sed -i "s#__POD_CONTAINERS_IMAGE__#registry.cn-hangzhou.aliyuncs.com/xhyimages/apiservice:${IMAGE_TAG_TO_INSTALL}#g" apiserver/deployjavaci.yaml
    - kubectl apply -f apiserver/deployjavaci.yaml || exit 1
  rules:
    - if: $CI_COMMIT_BRANCH == "main"
  tags:
    - study-runner

deploy_java_test:
  stage: deploy
  image: uhub.service.ucloud.cn/imxhy/kubectl:1.33.0
  when: manual
  script:
    - mkdir -p /.kube
    - echo $KUBE_CONFIG | base64 -d > $KUBECONFIG
    - kubectl version
    - IMAGE_TAG=$(echo "${CI_COMMIT_TIMESTAMP}" | sed 's/T/_/g; s/-//g; s/://g' | cut -c1-15)
    - IMAGE_TAG_TO_INSTALL=${CI_COMMIT_TAG:-$IMAGE_TAG}
    - echo $IMAGE_TAG_TO_INSTAL
    - sed -i "s#__POD_CONTAINERS_IMAGE__#registry.cn-hangzhou.aliyuncs.com/xhyimages/apiservice:${IMAGE_TAG_TO_INSTALL}#g" apiserver/deployjavatest.yaml
    - kubectl apply -f apiserver/deployjavatest.yaml || exit 1
  rules:
    - if: $CI_COMMIT_BRANCH == "main" || $CI_COMMIT_TAG
  tags:
    - study-runner

deploy_java_prod:
  stage: deploy
  image: uhub.service.ucloud.cn/imxhy/kubectl:1.33.0
  when: manual
  script:
    - mkdir -p /.kube
    - echo $KUBE_CONFIG | base64 -d > $KUBECONFIG
    - kubectl version
    - IMAGE_TAG=$(echo "${CI_COMMIT_TIMESTAMP}" | sed 's/T/_/g; s/-//g; s/://g' | cut -c1-15)
    - IMAGE_TAG_TO_INSTALL=${CI_COMMIT_TAG:-$IMAGE_TAG}
    - echo $IMAGE_TAG_TO_INSTAL
    - sed -i "s#__POD_CONTAINERS_IMAGE__#registry.cn-hangzhou.aliyuncs.com/xhyimages/apiservice:${IMAGE_TAG_TO_INSTALL}#g" apiserver/deployjavaprod.yaml
    - kubectl apply -f apiserver/deployjavaprod.yaml || exit 1
  rules:
    - if: $CI_COMMIT_TAG
  tags:
    - study-runner

check_java_ci_pod:
  stage: check
  image: uhub.service.ucloud.cn/imxhy/kubectl:1.33.0
  script:
    - mkdir -p /.kube
    - echo $KUBE_CONFIG | base64 -d > $KUBECONFIG
    - timeout 120 bash -c "until kubectl get pods -n gitlabci -l app=apiserver-ci --field-selector=status.phase=Running --no-headers | grep '1/1'; do sleep 3; done"
  rules:
    - if: $CI_COMMIT_BRANCH == "main"
  needs:
    - deploy_java_ci
  tags:
    - study-runner

check_java_test_pod:
  stage: check
  image: uhub.service.ucloud.cn/imxhy/kubectl:1.33.0
  script:
    - mkdir -p /.kube
    - echo $KUBE_CONFIG | base64 -d > $KUBECONFIG
    - timeout 120 bash -c "until kubectl get pods -n gitlabtest -l app=apiserver-test --field-selector=status.phase=Running --no-headers | grep '1/1'; do sleep 3; done"
  rules:
    - if: $CI_COMMIT_BRANCH == "main" || $CI_COMMIT_TAG
  needs:
    - deploy_java_test
  tags:
    - study-runner

check_java_prod_pod:
  stage: check
  image: uhub.service.ucloud.cn/imxhy/kubectl:1.33.0
  script:
    - mkdir -p /.kube
    - echo $KUBE_CONFIG | base64 -d > $KUBECONFIG
    - timeout 120 bash -c "until kubectl get pods -n gitlabprod -l app=apiserver-prod --field-selector=status.phase=Running --no-headers | grep '1/1'; do sleep 3; done"
  rules:
    - if: $CI_COMMIT_TAG
  needs:
    - deploy_java_prod
  tags:
    - study-runner

test:
  stage: test
  image: python:3.13.3
  before_script:
    - mkdir -p ~/.ssh
    - chmod 700 ~/.ssh
    - echo "$SSH_PRIVATE_KEY" > ~/.ssh/id_rsa
    - chmod 600 ~/.ssh/id_rsa
    - echo "Host ${GITLAB_HOST}" >> ~/.ssh/config
    - echo "    Port ${GITLAB_PORT}" >> ~/.ssh/config
    - echo "    User git" >> ~/.ssh/config
    - echo "    IdentityFile ~/.ssh/id_rsa" >> ~/.ssh/config
    - echo "    StrictHostKeyChecking no" >> ~/.ssh/config
    - ssh-keyscan -p ${GITLAB_PORT} ${GITLAB_HOST} >> ~/.ssh/known_hosts
  script:
    - git clone -b main ${REPO_URL}
    - cd autotest
    - python -m pip install --no-cache-dir -r requirements.txt
    - mkdir -p tests/reports
    - cd tests && pytest -s --junitxml=reports/report.xml || echo "Pytest exited with $?"
    - pwd
    - ls -l .
    - ls -l reports
  artifacts:
    reports:
      junit: autotest/tests/reports/report.xml
  rules:
    - if: $CI_COMMIT_BRANCH == "main"
  tags:
    - study-runner

前端webui项目部署

前端webui项目镜像构建

目录有所调整。

[root@gitclient mycicd]# vim webui/Dockerfile
FROM uhub.service.ucloud.cn/imxhy/node:23.11.0

MAINTAINER xhy@itzgr.com

RUN npm install -g @vue/cli

WORKDIR /opt/webui/

COPY . ./

RUN npm install

ENTRYPOINT ["npm","run","serve"]

创建webui项目部署文件

调整如下 ci 、 test 、 prod 部署文件。

ci 部署文件

[root@gitclient mycicd]# vim webui/deploywebuici.yaml
---
apiVersion: v1
kind: Namespace
metadata:
  name: gitlabci

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: deploy-webui-ci
  namespace: gitlabci
spec:
  replicas: 2
  revisionHistoryLimit: 5
  selector:
    matchLabels:
      app: webui-ci
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 25%
      maxSurge: 25%
  template:
    metadata:
      labels:
        app: webui-ci
    spec:
      containers:
        - name: webui-ci
          env:
            - name: TZ
              value: Asia/Shanghai
          image: __POD_CONTAINERS_IMAGE__
          imagePullPolicy: IfNotPresent
          ports:
          - containerPort: 8080
            protocol: TCP
          readinessProbe:
            httpGet:
              path: /
              port: 8080
              scheme: HTTP
            initialDelaySeconds: 30
            periodSeconds: 10
          livenessProbe:
            httpGet:
              path: /
              port: 8080
              scheme: HTTP
            initialDelaySeconds: 30
            periodSeconds: 10

---
apiVersion: v1
kind: Service
metadata:
  name: service-webui-ci
  namespace: gitlabci
spec:
  ports:
  - nodePort: 32111
    port: 8080
    protocol: TCP
    targetPort: 8080
  selector:
    app: webui-ci
  sessionAffinity: ClientIP
  type: NodePort

test 部署文件

[root@gitclient mycicd]# vim webui/deploywebuitest.yaml
---
apiVersion: v1
kind: Namespace
metadata:
  name: gitlabtest

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: deploy-webui-test
  namespace: gitlabtest
spec:
  replicas: 2
  revisionHistoryLimit: 5
  selector:
    matchLabels:
      app: webui-test
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 25%
      maxSurge: 25%
  template:
    metadata:
      labels:
        app: webui-test
    spec:
      containers:
        - name: webui-test
          env:
            - name: TZ
              value: Asia/Shanghai
          image: __POD_CONTAINERS_IMAGE__
          imagePullPolicy: IfNotPresent
          ports:
          - containerPort: 8080
            protocol: TCP
          readinessProbe:
            httpGet:
              path: /
              port: 8080
              scheme: HTTP
            initialDelaySeconds: 30
            periodSeconds: 10
          livenessProbe:
            httpGet:
              path: /
              port: 8080
              scheme: HTTP
            initialDelaySeconds: 30
            periodSeconds: 10

---
apiVersion: v1
kind: Service
metadata:
  name: service-webui-test
  namespace: gitlabtest
spec:
  ports:
  - nodePort: 32112
    port: 8080
    protocol: TCP
    targetPort: 8080
  selector:
    app: webui-test
  sessionAffinity: ClientIP
  type: NodePort

prod 部署文件

[root@gitclient mycicd]# vim webui/deploywebuiprod.yaml
---
apiVersion: v1
kind: Namespace
metadata:
  name: gitlabprod

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: deploy-webui-prod
  namespace: gitlabprod
spec:
  replicas: 2
  revisionHistoryLimit: 5
  selector:
    matchLabels:
      app: webui-prod
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 25%
      maxSurge: 25%
  template:
    metadata:
      labels:
        app: webui-prod
    spec:
      containers:
        - name: webui-prod
          env:
            - name: TZ
              value: Asia/Shanghai
          image: __POD_CONTAINERS_IMAGE__
          imagePullPolicy: IfNotPresent
          ports:
          - containerPort: 8080
            protocol: TCP
          readinessProbe:
            httpGet:
              path: /
              port: 8080
              scheme: HTTP
            initialDelaySeconds: 30
            periodSeconds: 10
          livenessProbe:
            httpGet:
              path: /
              port: 8080
              scheme: HTTP
            initialDelaySeconds: 30
            periodSeconds: 10

---
apiVersion: v1
kind: Service
metadata:
  name: service-webui-prod
  namespace: gitlabprod
spec:
  ports:
  - nodePort: 32113
    port: 8080
    protocol: TCP
    targetPort: 8080
  selector:
    app: webui-prod
  sessionAffinity: ClientIP
  type: NodePort

创建完整流水线

创建如下流水线，并且将前端 webui 项目的构建阶段全部整合到一起。
即对于前端 webui 项目，整合 build ----> deploy_webui_ci/deploy_webui_test ----> check_webui_ci_pod/check_webui_test_pod 全链路流程。

[root@gitclient mycicd]# vim webui/.gitlab-ci.yml
stages:
  - build
  - deploy
  - check

variables:
  KUBECONFIG: "/.kube/config"
  PROJECT_DIR: "${CI_PROJECT_DIR}/webui"

build_webui:
  stage: build
  image: uhub.service.ucloud.cn/imxhy/executor:v1.9.0-debug
  script:
    - cd ${PROJECT_DIR}
    - IMAGE_TAG=$(echo "${CI_COMMIT_TIMESTAMP}" | sed 's/T/_/g; s/-//g; s/://g' | cut -c1-15)
    - IMAGE_TAG_TO_INSTALL=${CI_COMMIT_TAG:-$IMAGE_TAG}
    - echo $IMAGE_TAG_TO_INSTALL
    - mkdir -p /kaniko/.docker
    - echo "{\"auths\":{\"registry.cn-hangzhou.aliyuncs.com\":{\"username\":\"${ALIYUN_USER}\",\"password\":\"${ALIYUN_PASSWORD}\"}}}" > /kaniko/.docker/config.json
    - cat /kaniko/.docker/config.json
    - echo ${PROJECT_DIR}
    - echo {\"username\":\"${ALIYUN_USER}\",\"password\":\"${ALIYUN_PASSWORD}\"}
    - ls ${PROJECT_DIR}/Dockerfile
    - >
      /kaniko/executor
      --context "${PROJECT_DIR}"
      --dockerfile "${PROJECT_DIR}/Dockerfile"
      --destination "registry.cn-hangzhou.aliyuncs.com/xhyimages/webui:${IMAGE_TAG_TO_INSTALL}"
      --registry-mirror "https://dbzucv6w.mirror.aliyuncs.com"
  rules:
    - if: $CI_COMMIT_BRANCH == "main" || $CI_COMMIT_TAG
  tags:
    - study-runner

deploy_webui_ci:
  stage: deploy
  image: uhub.service.ucloud.cn/imxhy/kubectl:1.33.0
  script:
    - mkdir -p /.kube
    - echo $KUBE_CONFIG | base64 -d > $KUBECONFIG
    - kubectl version
    - IMAGE_TAG=$(echo "${CI_COMMIT_TIMESTAMP}" | sed 's/T/_/g; s/-//g; s/://g' | cut -c1-15)
    - IMAGE_TAG_TO_INSTALL=${CI_COMMIT_TAG:-$IMAGE_TAG}
    - sed -i "s#__POD_CONTAINERS_IMAGE__#registry.cn-hangzhou.aliyuncs.com/xhyimages/webui:${IMAGE_TAG_TO_INSTALL}#g" webui/deploywebuici.yaml
    - kubectl apply -f webui/deploywebuici.yaml || exit 1
  rules:
    - if: $CI_COMMIT_BRANCH == "main"
  tags:
    - study-runner

deploy_webui_test:
  stage: deploy
  image: uhub.service.ucloud.cn/imxhy/kubectl:1.33.0
  when: manual
  script:
    - mkdir -p /.kube
    - echo $KUBE_CONFIG | base64 -d > $KUBECONFIG
    - kubectl version
    - IMAGE_TAG=$(echo "${CI_COMMIT_TIMESTAMP}" | sed 's/T/_/g; s/-//g; s/://g' | cut -c1-15)
    - IMAGE_TAG_TO_INSTALL=${CI_COMMIT_TAG:-$IMAGE_TAG}
    - sed -i "s#__POD_CONTAINERS_IMAGE__#registry.cn-hangzhou.aliyuncs.com/xhyimages/webui:${IMAGE_TAG_TO_INSTALL}#g" webui/deploywebuitest.yaml
    - kubectl apply -f webui/deploywebuitest.yaml || exit 1
  rules:
    - if: $CI_COMMIT_BRANCH == "main" || $CI_COMMIT_TAG
  tags:
    - study-runner

deploy_webui_prod:
  stage: deploy
  image: uhub.service.ucloud.cn/imxhy/kubectl:1.33.0
  when: manual
  script:
    - mkdir -p /.kube
    - echo $KUBE_CONFIG | base64 -d > $KUBECONFIG
    - kubectl version
    - IMAGE_TAG=$(echo "${CI_COMMIT_TIMESTAMP}" | sed 's/T/_/g; s/-//g; s/://g' | cut -c1-15)
    - IMAGE_TAG_TO_INSTALL=${CI_COMMIT_TAG:-$IMAGE_TAG}
    - sed -i "s#__POD_CONTAINERS_IMAGE__#registry.cn-hangzhou.aliyuncs.com/xhyimages/webui:${IMAGE_TAG_TO_INSTALL}#g" webui/deploywebuiprod.yaml
    - kubectl apply -f webui/deploywebuiprod.yaml || exit 1
  rules:
    - if: $CI_COMMIT_TAG
  tags:
    - study-runner

check_webui_ci_pod:
  stage: check
  image: uhub.service.ucloud.cn/imxhy/kubectl:1.33.0
  script:
    - mkdir -p /.kube
    - echo $KUBE_CONFIG | base64 -d > $KUBECONFIG
    - timeout 120 bash -c "until kubectl get pods -n gitlabci -l app=webui-ci --field-selector=status.phase=Running --no-headers | grep '1/1'; do sleep 3; done"
  rules:
      - if: $CI_COMMIT_BRANCH == "main"
  needs:
    - deploy_webui_ci
  tags:
    - study-runner

check_webui_test_pod:
  stage: check
  image: uhub.service.ucloud.cn/imxhy/kubectl:1.33.0
  script:
    - mkdir -p /.kube
    - echo $KUBE_CONFIG | base64 -d > $KUBECONFIG
    - timeout 120 bash -c "until kubectl get pods -n gitlabtest -l app=webui-test --field-selector=status.phase=Running --no-headers | grep '1/1'; do sleep 3; done"
  rules:
    - if: $CI_COMMIT_BRANCH == "main" || $CI_COMMIT_TAG
  needs:
    - deploy_webui_test
  tags:
    - study-runner

check_webui_prod_pod:
  stage: check
  image: uhub.service.ucloud.cn/imxhy/kubectl:1.33.0
  script:
    - mkdir -p /.kube
    - echo $KUBE_CONFIG | base64 -d > $KUBECONFIG
    - timeout 120 bash -c "until kubectl get pods -n gitlabprod -l app=webui-prod --field-selector=status.phase=Running --no-headers | grep '1/1'; do sleep 3; done"
  rules:
    - if: $CI_COMMIT_TAG
  needs:
    - deploy_webui_prod
  tags:
    - study-runner

构建父子类型流水线

父子类型流水线适用于将多个子项目放在一个项目代码仓中的场景。
该场景中，流水线中存在后端 Java 、前端 webui 两个项目的编译、构建、发布以及集成在一起的测试和部署等。
因此可以使用父子类型的流水线，且配置是如果只修改了Java子项目，则只执行和Java子项目相关的流水线任务即可。

前端子项目的目录为 webui，后端子项目的目录为apiserver，组建父子类型的流水线，就是在项目的根目录创建流水线发布文件.gitlab-ci.yml作为父流水线的配置，各子项目分别创建一个.gitlab-ci.yml作为子流水线的位置。

如上所致 java 后端和 webui 前端流水线均创建完成。

父流水线
子流水线不会自动触发，需要项目根目录下的.gitlab-ci.yml文件进行触发。
在父流水线中调用子流水线需要使用关键字 trigger 和 include 。

[root@gitclient mycicd]# vim .gitlab-ci.yml
stages:
  - triggers
  - pre-check

verify_files:
  stage: pre-check
  script:
    - ls -l apiserver/.gitlab-ci.yml
    - ls -l webui/.gitlab-ci.yml

trigger_apiserver:
  stage: triggers
  trigger:
    include: apiserver/.gitlab-ci.yml
    forward:
      pipeline_variables: true
  variables:
    PARENT_TAG: $CI_COMMIT_TAG
  rules:
    - if: $CI_COMMIT_BRANCH == "main"
      changes:
        - apiserver/*
    - if: $CI_COMMIT_TAG
      changes: []

trigger_webui:
  stage: triggers
  trigger:
    include: webui/.gitlab-ci.yml
    forward:
      pipeline_variables: true
  variables:
    PARENT_TAG: $CI_COMMIT_TAG
  rules:
    - if: $CI_COMMIT_BRANCH == "main"
      changes:
        - webui/*
    - if: $CI_COMMIT_TAG
      changes: []

如上定义了一个节点， trigger ，然后 trigger 里定义了两个任务，通过 include 关键字将子目录下的 .gitlab-ci.yml 引用。
rules 和 changes 关键字是用来控制 job 触发执行的，指定具体的目录。
如上 trigger_back 的 job 指定了检测目录为 apiserver，即只有当 apiserver 目录中的文件或代码发生了变化，apiserver 目录中的子流水线才会执行。

提交流水线
首次提交后，由于前后端都是第一次生成，因此会自动执行完所有任务。

[root@gitclient mycicd]# git add .
[root@gitclient mycicd]# git status 

[root@gitclient mycicd]# git commit -m "Deploy apiserver and webui cicd"
[root@gitclient mycicd]# git push origin main

查看流水线

查看提交流水线后的作业情况。

215

216
217

确认验证

部署情况确认

查看部署在 Kubernetes 后的应用，

root@master01:~# kubectl -n gitlabci get pods -o wide
NAME                                   READY   STATUS    RESTARTS   AGE    IP             NODE       NOMINATED NODE   READINESS GATES
deploy-apiserver-ci-68655894c8-qzcwr   1/1     Running   0          117m   10.10.19.82    worker03   <none>           <none>
deploy-apiserver-ci-68655894c8-vrdwd   1/1     Running   0          117m   10.10.5.47     worker01   <none>           <none>
deploy-webui-ci-5cfb75dfcf-7zm8w       1/1     Running   0          111m   10.10.19.115   worker03   <none>           <none>
deploy-webui-ci-5cfb75dfcf-zncr9       1/1     Running   0          111m   10.10.5.35     worker01   <none>           <none>
root@master01:~# kubectl -n gitlabci get svc -o wide
NAME                   TYPE       CLUSTER-IP     EXTERNAL-IP   PORT(S)          AGE    SELECTOR
service-apiserver-ci   NodePort   10.20.29.161   <none>        8080:32101/TCP   117m   app=apiserver-ci
service-webui-ci       NodePort   10.20.10.106   <none>        8080:32111/TCP   112m   app=webui-ci
root@master01:~# kubectl -n gitlabci get deployments.apps
NAME                  READY   UP-TO-DATE   AVAILABLE   AGE
deploy-apiserver-ci   2/2     2            2           117m
deploy-webui-ci       2/2     2            2           112m

验证后端应用

浏览器直接访问： http://172.24.8.180:32101/demo/hello 。

218

验证前端应用

浏览器直接访问： http://172.24.8.180:32111
219

Ddevops梳理

如上所示为一个包含前后端的完整流水线，其主要包括过程总结如下：

当后端研发人员提交代码后，流水线会自动触发。首先执行compile阶段，主要包括unittest_java、compile_java和静态代码检查sonarqube_check_java，这三个认为是并行执行的。而且只要compile阶段执行完成，不论单元测试和静态代码检查是否执行完成，都会执行第二个build镜像构建阶段。当build镜像构建阶段完成后，自动部署到CI环境，CI环境部署完成后，开始检查CI环境中的Pod状态是否正常。检查完成后，开始自动执行自动化测试。当自动化测试执行完成后，根据测试结果及其他需求，计划是否要部署到测试环境，测试环境的部署由手动执行。
部署到测试环境后，流水线执行如下，然后测试团队可以对测试环境进行测试：

221

当测试团队测试完成，并对测试结果进行bug修复完成后，可以通过tags部署线上生产环境。

222

打完tag后同样会触发新的流水线，通过打tag触发的流水线有两个选择，即部署测试环境或部署生产环境，同时保留手动执行，即在通过CI环境同时修复bug后打tag，然后人为审核是否发版，然后手动执行。
手动部署 test 环境，然后可以通知测试人员对test环境进行测试。
测试团队确认后，可以手动开始部署 prod 生产环境。