GitLAB CI-CD语法

GitLAB CI-CD语法

image-20230508193755979

目录

image-20230518111044933

1、Pipeline核心语法

gitlab-ci语法:

https://docs.gitlab.com/ee/ci/yaml/

image-20230508195014808

stages 阶段控制

  • .pre阶段的作业总是在流水线开始时执行;
  • .post阶段的作业总是在流水线结束时执行;

CI代码:

stages:
  - build
  - test
  - deploy 

job0:
  tags:
    - go
  stage: .pre
  script:
    - echo " init"

job1:
  tags:
    - go
  stage: build
  script:
    - echo "build"

job2-1:
  tags:
    - mvn
  stage: test
  script:
    - echo "test"
    - sleep 10

job2-2:
  tags:
    - mvn
  stage: test
  script:
    - echo "test"

job3:
  tags:
    - go
  stage: deploy
  script:
    - echo "deploy"

job10:
  tags:
    - go
  stage: .post
  script:
    - echo "end"

代码直接写在gitlab的CI/CD的editor里:

image-20230508200801392

  • 这里我们先修改下runner上的标签

image-20230508200030139

image-20230508200055459

  • 运行

image-20230508200238359

如果两个或者多个作业,指向同一个阶段名称,则该阶段下的所有作业都并行运行;如果不能并行运行,需要检查runner的配置文件中的concurrent值, 要大于1。

image-20230508200545635

variables 环境变量

变量可以分为全局变量和局部变量;全局变量是整个流水线可以用的,局部变量是仅在作业中生效的;

  • CI代码
stages:
  - build

variables:
  BUILD_TOOLS: 
    value: "mvn"
    description: "choice build tools"   #描述信息,注释不能用//,只能用#号。
  RUNNER_TAG: "go"


job1:
  tags:
    - "${RUNNER_TAG}"
  stage: build
  variables:
    BUILD_TOOLS: "gradle"
  script:
    - echo "${BUILD_TOOLS}"
  • 运行

image-20230508201543785

image-20230508201601217

job 作业默认配置

定义一个作业的时候,一般定义哪些关键字呢? 作业在哪个runner运行? 作业属于流水线中的哪个阶段? 这个作业要做什么?

  • CI代码
stages:
  - build

variables:
  RUNNER_TAG: "go"

before_script:
  - echo "pipeline before script"
after_script:
  - echo "pipeline after script"

job1:
  tags:
    - ${RUNNER_TAG}
  stage: build
  before_script:
    - echo "before script...."
  script:
    - echo "mvn package"
  after_script:
    - echo "after script..."

job2:
  tags:
    - ${RUNNER_TAG}
  stage: build
  script: 
    - echo "build"

参数解析:

语法关键字作用备注
variables定义作业中的环境变量;
tags根据标签选择运行作业的构建节点;多个标签, 则匹配具有所有标签的构建节点;GitLab14.1版本后, 标签的值可以使用变量;GitLab14.3版本后, 标签数量必须小于50;
stage指定当前作业所属的阶段名称;
before_script作业在运行前执行的Shell命令行;
script作业在运行中执行的Shell命令行;每个作业至少要包含一个script;
after_script作业在运行后执行的Shell命令行;
  • 运行

image-20230509073124496

image-20230509073217401

image-20230509073245729

需要注意:如果before_script和after_script定义在pipeline里,则每个作业里都会运行这2个脚本;如果是定义在某个job里,则只会在其job里运行!

job 作业运行控制

语法关键字作用备注
allow_failure控制作业状态,是否允许作业失败,默认值为false 。启用后,如果作业运行失败,该作业将在用户界面中显示橙色警告。[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-vro2Lluq-1684397903970)(https://bucket-hg.oss-cn-shanghai.aliyuncs.com/img/1630506004511-aa8a03a2-c6b7-4c7e-b2a6-9ef2725c5fd2.png)]管道将认为作业成功/通过,不会被阻塞。 假设所有其他作业均成功,则该作业的阶段及其管道将显示相同的橙色警告。但是,关联的提交将被标记为"通过",而不会发出警告。
when根据状态控制作业运行, 当前面作业成功或者失败时运行。on_success 前面阶段成功时执行(默认值);on_failure 前面阶段失败时执行;always 总是执行;manual 手动执行;delayed 延迟执行;start_in’5’5 seconds30 minutes1 day1 weeknever 永不执行;
retry作业重新运行,遇到错误重新运行的次数。值为整数等于或大于0,但小于或等于2异常分类 (一般就是3次)
timeout作业运行超时时间;
rules根据特定的变量或文件变更来控制作业运行;ifchangesexists
needs作业依赖控制;needs: [“作业名称”]
parallel生成多个作业,并行运行parallel:5值 2-50之间
parallel 并行运行
  • 代码
stages:
  - build

variables:
  RUNNER_TAG: "go"

job1:
  tags:
    - ${RUNNER_TAG}
  stage: build
  parallel: 5
  script:
    - echo "mvn package"

模拟做压测。

  • 运行

image-20230510204224755

image-20230510204317770

needs 作业关联运行
  • CI代码

needs 指定要依赖的作业名称

stages:
  - build
  - test

variables:
  RUNNER_TAG: "go"

job1:
  tags:
    - ${RUNNER_TAG}
  stage: build
  script:
    - echo "mvn package"
    - sleep 10

job2:
  tags:
    - ${RUNNER_TAG}
  stage: build
  script: 
    - echo "build"


job3:
  tags:
    - ${RUNNER_TAG}
  stage: test
  script:
    - echo "mvn package"

job4:
  tags:
    - ${RUNNER_TAG}
  stage: test
  needs: ["job2"]
  script:
    - echo "mvn package"



  • 运行

tstmp_20230510204814

tstmp_20230510204829

rules根据变量/文件控制
根据条件(变量)判断: IF

用法:

定义变量条件;

运算符:
● =!==~

条件链接符:
● &&||

CI代码:

variables:
  DOMAIN: example.com

codescan:
  stage: build
  tags:
    - build
  rules:
    - if: '$DOMAIN == "example.com"'
      when: manual
    - when: on_success
  script:
    - echo "codescan"
    - sleep 5;

运行:

image-20230510221518531

image-20230510221530477

image-20230510221543129

根据文件判断: changes exists

changes

文件变更时条件为真;

示例: 代码提交后,当Dockerfile文件发生变更则条件为真。即手动执行此作业。

img

注意:非Push相关动作,此条件永远为真。

CI代码:

citest1:
  tags:
    - build
  stage: test
  rules:
    - changes:
        - Dockerfile
      when: manual
    - when: never
  script:
    - echo "Do a test here"
    - echo "For example run a test suite"

运行:

直接运行次作业时,肯定是不会运行流水线的,因此此时是没有Dockerfile文件的。

我们来创建一个Dockerfile文件,再次运行流水线,并观察现象:

image-20230510222132121

image-20230510222138402

allow_failure

允许作业在不停止管道的情况下失败;

CI代码:

stages:
  - test

citest1:
  tags:
    - build
  stage: test
  allow_failure: true
  script:
    - lsl -l
    - echo "Do a test here"


citest2:
  tags:
    - build
  stage: test
  script:
    - ls -l
    - echo "Do a test here"
 

运行:

image-20230511213644637

image-20230511213721309

image-20230511213731113

variables

重新定义变量的值;

示例: 如果当前的分支是主干分支main, 则重新定义ENV_TYPE的值为prod

img

注意: 只影响局部作业内的变量,不会影响全局变量。

CI代码:

variables:
  ENV_TYPE: "dev"
  
cddeploy:
  tags:
    - build
  stage: deploy
  rules:
    - if: $CI_COMMIT_REF_NAME == "main"
      variables:
        ENV_TYPE: "prod"
  script:
    - echo "Deploy env ${ENV_TYPE}"  

运行:

image-20230511214046220

🍀 某一类文件写法:

image-20230511214327721

when

同when语法的运行方式;

这里的when是写在rules里面的,也可以写在外面的。

rules代码汇总:

stages:
  - build
  - test

variables:
  RUNNER_TAG: "go"
  SKIP_BUILD: "true"


job1:
  tags:
    - ${RUNNER_TAG}
  stage: build
  rules:
    - if: '$SKIP_BUILD == "true"'   # 此时job1 不会被运行
      when: never
    - when: always
  before_script:
    - echo "before script...."
  script:
    - echo "mvn package"
  after_script:
    - echo "after script..."


job2:
  tags:
    - ${RUNNER_TAG}
  stage: test
  rules:
    - changes:
      - README.md     # 提交时仅该文件存在变更时才会触发
      when: always
    - when: never
  script:
    - echo "test"

job3:
  tags:
    - ${RUNNER_TAG}
  stage: test
  rules:
    - exists:
        - Dockerfile  #只要项目中存在该文件就会运行,提交存在也算。
      when: manual
    - when: never
  script:
    - echo "test"

job4:
  tags:
    - ${RUNNER_TAG}
  stage: test
  script:
    - echo "test"

运行:

image-20230512072041799

when 状态控制和运行方式
  • 根据上游作业的状态决定
    • 当前作业是否运行?
    • 运行的方式?(手动/自动/定时)

CI代码:

stages:
  - build
  - test
  - deploy

variables:
  RUNNER_TAG: "go"

job1:
  tags:
    - ${RUNNER_TAG}
  stage: build
  before_script:
    - echo "before script...."
  script:
    - echo "mvn package"
  after_script:
    - echo "after script..."



job2:
  tags:
    - ${RUNNER_TAG}
  stage: test
  when: on_success
  script: 
    - echo "build"


job3:
  tags:
    - ${RUNNER_TAG}
  stage: test
  when: on_failure
  script: 
    - echo "build"

job4:
  tags:
    - ${RUNNER_TAG}
  stage: deploy
  when: manual
  script: 
    - echo "build"

运行:

image-20230512072456135

when: delay

test1:
  tags:
    - ${RUNNER_TAG} 
  stage: test
  script:
    - echo "Do a test here"
    - echo "For example run a test suite"
    - sleep 3
  when: delay
  start_in: '5'

image-20220506121738112

timeout作业运行超时时间
build:
  script: build.sh
  timeout: 3 hours 30 minutes

test:
  script: rspec
  timeout: 3h 30m
retry 作业失败后重试次数

CI代码:

stages:
  - build
  - test

variables:
  RUNNER_TAG: "go"

job1:
  tags:
    - ${RUNNER_TAG}
  stage: build
  retry: 2
  before_script:
    - echo "before script...."
  script:
    - echo "mvn package"
    - mvs
  after_script:
    - echo "after script..."

job2:
  tags:
    - ${RUNNER_TAG}
  stage: test
  script: 
    - echo "build"

运行:

image-20230512073129706

根据特定的错误匹配:

always :在发生任何故障时重试(默认)。
unknown_failure :当失败原因未知时。
script_failure :脚本失败时重试。
api_failure :API失败重试。
stuck_or_timeout_failure :作业卡住或超时时。
runner_system_failure :构建节点的系统发生故障。
missing_dependency_failure: 依赖丢失。
runner_unsupported :Runner不受支持。
stale_schedule :无法执行延迟的作业。
job_execution_timeout:作业运行超时。
archived_failure :作业已存档且无法运行。
unmet_prerequisites :作业未能完成先决条件任务。
scheduler_failure :调度失败。
data_integrity_failure :结构完整性问题。


####
max :最大重试次数  when :重试失败的错误类型

stages:
  - build
  - test

variables:
  RUNNER_TAG: "go"

job1:
  tags:
    - ${RUNNER_TAG}
  stage: build
  retry: 2  # 不管任何错误,都重试2次
  before_script:
    - echo "before script...."
  script:
    - echo "mvn package"
    - mvs  # 命令错误
  after_script:
    - echo "after script..."

job2:
  tags:
    - ${RUNNER_TAG}
  stage: test
  when: on_failure  # 为了让这个作业运行所以添加的,不然前面作业失败这个作业就不运行了。
  script: 
    - echo "build"
    - aaa   # 命令错误
  retry:
    max: 2
    #when: api_failure
    when: script_failure   # 定义脚本错误:  正常会retry两次

运行:

image-20230512073329001

allow_failure 允许作业失败

CI代码:

stages:
  - build
  - test

variables:
  RUNNER_TAG: "go"

job1:
  tags:
    - ${RUNNER_TAG}
  stage: build
  allow_failure: true
  retry: 2
  before_script:
    - echo "before script...."
  script:
    - echo "mvn package"
    - mvs
  after_script:
    - echo "after script..."

job2:
  tags:
    - ${RUNNER_TAG}
  stage: test
  script: 
    - echo "build"

运行:

image-20230512073710528

2、Pipeline运行控制

image-20230512073922712

1.workflow 控制流水线

控制管道是否创建和运行。根据条件(变量)判断: IF

  • if 定义变量条件;

  • variables 重新定义变量的值;

  • when

    • always
    • never
命令备注
if: ‘$CI_PIPELINE_SOURCE == “merge_request_event”’合并请求时运行流水线;
if: ‘$CI_PIPELINE_SOURCE == “push”’提交代码运行流水线;
  • 预定义变量

image-20230512074126286

https://docs.gitlab.com/ee/ci/variables/predefined_variables.html

image-20230512074438057

🍀 demo

CI代码:

variables:
  SKIP_RUN: "true"
  RUNNER_TAG: "go"

workflow:
  rules:
    - if: $CI_PIPELINE_SOURCE == "push"
      when: never
      
stages:
  - build

job2:
  tags:
    - ${RUNNER_TAG}
  stage: build
  script: 
    - echo "build"

运行:

运行后,是没看见触发流水线的。

🍀 demo

CI代码:

variables:
  SKIP_RUN: "true"
  RUNNER_TAG: "go"

workflow:
  rules:
    - if: $CI_PIPELINE_SOURCE == "push"
      when: never
    - if: $CI_PIPELINE_SOURCE == "web"
      when: never
      
stages:
  - build

job2:
  tags:
    - ${RUNNER_TAG}
  stage: build
  script: 
    - echo "build"

运行:

可以看到,这里在web触发时,依然报错了:……

image-20230512075826983

2.Git 选项跳过流水线

  • 提交信息中添加关键字 [ci skip] 或者 [skip ci]
  • Git 2.10 更高版本,可以通过以下配置设置CI/CD;
## 跳过
git push -o ci.skip

## 传递变量
git push -o ci.variable="MAX_RETRIES=10" -o ci.variable="MAX_TIME=600"

🍀 demo

这里改变下项目里readme文件内容,进行提交,commit message信息里写上[ci skip],再次观察是否会触发提交流水线(默认是提交一次就会触发一次流水线的)?

image-20230513075150576

image-20230513075216517

可以看到,这里跳过了流水线的触发。

3.trigger 触发下游管道

  • 触发项目管道
  • 触发子管道

strategy: 默认情况下,一旦创建了下游管道,trigger作业就会以success状态完成。要强制等待下游管道完成,使用 strategy: depend

触发项目管道:(不要自己触发自己,这样就进入到一个环里面了。)

triggers:
  stage: deploy
  trigger:
    project: devops/devops-maven-service
    branch: main
    strategy: depend    ## 状态同步

触发子管道:

stages:
  - deploy

trigger-cd-pipeline:   
  stage: deploy
  trigger:
    include: ci/stages.yml  ## 触发当前项目的子流水线


trigger-project-pipeline:
  stage: deploy
  trigger:
    include:
    - project: "devops/devops05-app-service"   ## 触发其它项目的子流水线
      ref: "main"
      file: "ci/stages.yml"

tstmp_20230513081626

测试过程:

  • 创建一个项目

devops6-gitlabci-demo

image-20230513081821871

  • 在新项目里创建一个.gitllabci.yml文件:

image-20230513082056693

stages:
  - build

before_script:
  - echo "Before script section"
  - echo "For example you might run an update here or install a build dependency"
  - echo "Or perhaps you might print out some debugging details"

after_script:
  - echo "After script section"
  - echo "For example you might do some cleanup here"

build1:
  tags:
    - go
  stage: build
  script:
    - sleep 20
    - echo "Do your build here"
  • 先来测试下 触发项目管道

image-20230513082438122

stages:
  - deploy

triggers:
  stage: deploy
  trigger:
    project: devops6/devops6-gitlabci-demo
    branch: main
    strategy: depend    ## 状态同步

提交代码并观察现象:

image-20230513082532404

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-57uSuqHG-1684397903079)(https://bucket-hg.oss-cn-shanghai.aliyuncs.com/img/image-20230513082635830.png)]

image-20230513082618952

可以看到,此时成功触发了其他项目。

  • 再来测试下 trigger触发当前项目的子流水线

在当前项目devops-demo-service里创建.ci/ci.yaml文件:

image-20230513094956436

stages:
  - build
  - test

build01:
  stage: build
  tags:
    - go
  script:
    - echo "build"


test01:
  stage: test
  tags:
    - go
  script:
    - echo "test"

提交后,默认就会触发流水线的运行:

image-20230513095135054

此时,有这个需求:改了目录里面的内容,就不让他构建,该怎么写?

CI代码:

stages:
  - deploy

workflow:
  rules:
    - changes:
        - .ci/*
      when: never
    - when: always

triggers:
  stage: deploy
  trigger:
    project: devops6/devops6-gitlabci-demo
    branch: main
    strategy: depend    ## 状态同步

image-20230513095429901

此时,提交后,我们再次修改.ci/ci.yaml内容,再次观察是否会触发流水线?

image-20230513095548044

stages:
  - build
  - test

build01:
  stage: build
  tags:
    - go
  script:
    - echo "build"


test01:
  stage: test
  tags:
    - go
  script:
    - echo "test"

test02:
  stage: test
  tags:
    - go
  script:
    - echo "test"

提交并观察:

image-20230513095627041

可以看到,此时就不会触发流水线了。

我们继续看下trigger如何触发当前项目的子流水线

CI代码:

stages:
  - deploy

workflow:
  rules:
    - changes:
        - .ci/*
      when: never
    - when: always

triggers:
  stage: deploy
  trigger:
    project: devops6/devops6-gitlabci-demo
    branch: main
    strategy: depend    ## 状态同步

triggers2:
  stage: deploy
  trigger:
    include: .ci/ci.yaml

image-20230513095856577

提交:

image-20230513095945607

此时,已经成功触发了当前项目的子流水线。

  • 我们再来看看如何触发其他项目的子流水线?

在之前新创建的devops6-gitlabci-demo项目里创建.ci/ci.yaml

image-20230513100150410

stages:
  - build
  - test

build01:
  stage: build
  tags:
    - go
  script:
    - echo "build"


test01:
  stage: test
  tags:
    - go
  script:
    - echo "test"

test02:
  stage: test
  tags:
    - go
  script:
    - echo "test"

提交。
然后在devops-demo-service项目的.gitlab-ci.yml里编写CI代码:

image-20230513100447162

stages:
  - deploy

workflow:
  rules:
    - changes:
        - .ci/*
      when: never
    - when: always

triggers:
  stage: deploy
  trigger:
    project: devops6/devops6-gitlabci-demo
    branch: main
    strategy: depend    ## 状态同步

triggers2:
  stage: deploy
  trigger:
    include: .ci/ci.yaml

trigger-project-pipeline:
  stage: deploy
  trigger:
    include:
    - project: "devops6/devops6-gitlabci-demo"   ## 项目名称
      ref: "main"
      file: ".ci/ci.yaml"   

提交并观察现象:

image-20230513100526091

符合预期。

4.API触发Pipeline

image-20230513102135105

创建一个认证token:

image-20230513102225902

使用API触发

使用curl命令进行测试:

curl -X POST \
     -F token=156d70fce2d6afca6d0b21c6624b85 \
     -F ref=main \
     http://172.29.9.101:8076/api/v4/projects/2/trigger/pipeline

这里的2是项目id:

image-20230513102404172

触发后,报如下提示:

image-20230513102457749

看是被workflow规则过滤了:

这里再改下CI代码:

image-20230513102727532

提交后,再次在命令行进行触发:

image-20230513102801542

这里还是提示被workflow规则过滤了。

再次修改下CI代码:

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-uOJvs0mZ-1684397903083)(https://bucket-hg.oss-cn-shanghai.aliyuncs.com/img/image-20230513102849535.png)]

stages:
  - deploy

workflow:
  rules:
    - if: $CI_PIPELINE_SOURCE == "trigger"
      when: always          
    - changes:
        - .ci/*
      when: never
    - when: always

triggers:
  stage: deploy
  trigger:
    project: devops6/devops6-gitlabci-demo
    branch: main
    strategy: depend    ## 状态同步

triggers2:
  stage: deploy
  trigger:
    include: .ci/ci.yaml

trigger-project-pipeline:
  stage: deploy
  trigger:
    include:
    - project: "devops6/devops6-gitlabci-demo"   ## 项目名称
      ref: "main"
      file: ".ci/ci.yaml"   

提交后,再次在命令行触发,观察现象:

image-20230513103034120

image-20230513103049747

因此,要特别注意下,这个workflow下的规则匹配。(这里把它写在最上面就可以正常触发了,但是写在changes下面,就不能正常触发的)

使用postman测试

tstmp_20230513163849

gitlabci作业中触发

script:
  - "curl -X POST -F token=TOKEN -F ref=REF_NAME http://172.29.9.101:8076/api/v4/projects/2/trigger/pipeline"

优化: 将token以变量的方式存储到项目中。

stages:         
  - build

build-job: 
  tags:
    - build     
  stage: build
  script:
    - "curl -X POST -F token=${CITOKEN} -F ref=main  http://172.29.9.101:8076/api/v4/projects/2/trigger/pipeline"
    - echo "Compile complete."
    - sleep 1
触发并传递参数

     curl -X POST \
     -F token=156d70fce2d6afca6d0b21c6624b85 \
     -F ref=main \
     -F "variables[BUILD_TOOL]=mavenandmaven" \
     http://172.29.9.101:8076/api/v4/projects/2/trigger/pipeline     
ciTriggerTest:
  tags:
    - build
  stage: build
  script:
    - echo ${BUILD_TOOL}

image-20230513165539695

3、PipelineTemplate实践

https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates

extends

extends可以指定多个模板, 相同的参数最后模板会覆盖前面的。例如: 下面两个模板中,继承关系是:

  • 继承 .test1 (此时rspec的变量NAME的值为gitlab)
  • 继承 .test2 (此时rspec的变量NAME的值为gitlabCI , 覆盖了.test1中的值)
.test1:
  variables:
    NAME: "gitlab"
  tags:
    - build
  stage: test
  rules:
    - if: $CI_COMMIT_BRANCH == "main"
  script: echo "mvn test"

.test2:
  variables:
    NAME: "gitlabCI"
  tags:
    - build01
  stage: test
  
rspec:
  extends: 
    - .test1
    - .test2
  script: echo " DevOps"

      
      
      
###### 结果

rspec:
  variables:
    NAME: "gitlabCI"
  tags:
    - build01
  stage: test
  rules:
    - if: $CI_COMMIT_BRANCH == "main"
  script: echo " DevOps"   

include

include用于在CI/CD 配置中引入外部 YAML 文件。可以将一个长.gitlab-ci.yml文件分解为多个文件以提高可读性,或减少同一配置在多个地方的重复。

  • local 导入当前仓库中的文件;
  • file 导入当前项目或其他项目库中的文件;
  • remote 导入一个远程的文件(例如:http://xxx.yaml)
  • template 导入GitLab官方提供的模板文件;
## 本地仓库文件
include:
  - local: '/templates/.gitlab-ci-java.yml'
 
## 其他仓库文件
include:
  - project: 'devops/my-project'
    ref: main
    file: 
      - '/templates/.gitlab-ci-java.yml'
      - '/templates/.tests.yml'
    
## 远程文件
include:
  - remote: 'https://192.168.1.200//-/raw/main/.gitlab-ci.yml'

🍀 测试

Extend:继承模板

Include:引入模板

  • 创建新项目 gitlab-temp-repo

image-20230514094405077

  • 定义2个模板

创建CI/java-ci.yaml文件:

image-20230514171303999

.test1:
  variables:
    NAME: "gitlab"
  tags:
    - build
  stage: test
  rules:
    - if: $CI_COMMIT_BRANCH == "main"
  script: echo "mvn test"

.test2:
  variables:
    NAME: "gitlabCI"
  tags:
    - build02
  stage: test
  script: echo "mvn test"  
  • devops-demo-service项目编辑CI文件

image-20230514171800151

include:
  - project: 'devops6/gitlab-temp-repo'
    ref: main
    file: 
      - '/CI/java-ci.yaml'

test1:
  extends:
    - .test1

test2:
  tags:
    - build
  extends:
    - .test2    
  • 运行

image-20230514171734456

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-OheUjmKR-1684397903084)(https://bucket-hg.oss-cn-shanghai.aliyuncs.com/img/image-20230514171852495.png)]

能够做成每个项目一个template就很不错了。

综合实例:

将模板文件,放到单独的一个仓库中;

ci.yml

.build:
  tags:
    - build
  stage: build
  variables:
    BUILD_TOOL: "maven3"
    BUILD_SHELL: "mvn clean package"
  script:
    - "${BUILD_SHELL}"

.test:
  tags:
    - "linux"
  stage: test
  variables:
    TEST_SHELL: "mvn test"
  script:
    - "${TEST_SHELL}"

.gitlab-ci.yml

workflow:
  rules:
    - if: $CI_COMMIT_BEFORE_SHA == "0000000000000000000000000000000000000000"  # 40个零 创建分支或者tag
      when: never

include:
  - project: 'devops/devops-ci-lib'   ## 引入ci.yml模板
    ref: main
    file: '/CI/ci.yml'

variables:
  GIT_CHECKOUT: "false"
  RUNNER_TAG: "mvn"
  BUILD_SHELL: "mvn clean package -DskipTests"
  TEST_SHELL: "mvn test"

stages:
  - build 
  - test

# 下载代码
checkout:
  tags:
    - ${RUNNER_TAG}
  stage: .pre 
  variables:
    GIT_CHECKOUT: "true"
  script:
    - echo "GetCode..."

# maven构建
build:
  tags:
    - ${RUNNER_TAG}
  extends: .build  ##重用build作业

# maven test
test:
  tags:
    - ${RUNNER_TAG}
  extends: .test

💘 实践:练习Pipeline模板库(测试成功)-2022.5.6(这个有点难度)

🍀 老师文档

  1. 创建一个新的项目 “devops4/devops4-devops-service” (模板库);
  2. 为项目添加一个CI模板文件(ci/ci.yml);(参考pipeline.yml)
  3. 添加cicd.yml 文件, pipeline, include导入模板库中的ci/ci.yml

image-20220506233150046

ci/ci.yml: ‘devops4/devops4-devops-service 模板库

.build:
  variables:
    SHELL: "mvn clean build"
  tags:
    - build
  stage: build
  script: echo "${SHELL}"

cicd.yml

### 导入仓库文件
include:
  - project: 'devops4/devops4-devops-service'
    ref: main
    file: 
      - '/ci/ci.yml'
variables:
  BUILD_SHELL: "mvn build"
stages:
  - build
  
### 继承模板
build:
  extends: 
    - .build
  variables:
    SHELL: $BUILD_SHELL

远程CI文件配置:

http://192.168.1.200/devops4/devops4-devops-service/-/raw/main/cicd.yml

image-20220506233706473

🍀 自己测试过程:

  • 创建模板库devops4-devops-service

image-20220507002910165

创建如下相关文件:

ci/ci.yml

.build:
  variables:
    SHELL: "mvn clean build"
  tags:
    - build
  stage: build
  script: echo "${SHELL}"

cicd.yml

### 导入仓库文件
include:
  - project: 'devops4/devops4-devops-service'
    ref: main
    file: 
      - '/ci/ci.yml'
      
variables:
  BUILD_SHELL: "mvn build"
stages:
  - build
  
### 继承模板
build:
  extends: 
    - .build
  variables:
    SHELL: $BUILD_SHELL

image-20220507003032670

写完后提交。

获取cicd.yml文件的路径:

image-20220507003101128

http://172.29.9.101/devops4/devops4-devops-service/-/raw/main/cicd.yml

image-20220507003120723

  • 来到另一个项目里devops4-monrepo-service

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-4fGskq6r-1684397903086)(https://bucket-hg.oss-cn-shanghai.aliyuncs.com/img/image-20220506235944620.png)]

在项目>Settings>CI/CD>Gerneral pipelines里把刚才拷贝的那个链接填在这里,保存:

image-20220507002618326

  • 然后,直接运行这个流水线,观察效果:

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-lhnnpHbM-1684397903086)(https://bucket-hg.oss-cn-shanghai.aliyuncs.com/img/image-20220507003218528.png)]

image-20220507003237384

符合预期,实验结束。

4、提交阶段流水线

因为GitLabCI与GitLab版本控制系统深度集成,所以不需要配置触发器。

  • 默认已经支持了提交代码、合并代码触发流水线的运行;
  • 默认流水线运行后会自动的下载本项目代码;(每个job会下载项目代码)

流水线优化

1.过滤新建分支和tag的触发
workflow:
  rules:
    - if: $CI_COMMIT_BEFORE_SHA == "0000000000000000000000000000000000000000"
      when: never

🍀 测试过程如下:

  • 先来测试下,gitlabci里新建分支是否会触发流水线的运行?

本次在devops4-commit-service项目里测试:

新创建分支feature-dev-03

image-20220513104839095

可以看到,新建分支操作是会触发流水线运行的:

image-20220513104931751

  • 编写pipeline代码,通过CI_COMMIT_BEFORE_SHA来过滤新建分支和tag的触发:

编写之前可以来看下gitlabci官网预定义的一些变量:

image-20220513105305340

image-20220513105411089

workflow:
  rules:
    - if: $CI_COMMIT_BEFORE_SHA == "0000000000000000000000000000000000000000"
      when: never
    - when: always

stages:
  - build 
  - test
  - deploy
before_script:
  - echo "Before script section"
  - echo "For example you might run an update here or install a build dependency"
  - echo "Or perhaps you might print out some debugging details"

after_script:
  - echo "After script section"
  - echo "For example you might do some cleanup here"

build:
  stage: build
  script:
    - echo "Do your build here"

test:
  stage: test
  script:
    - echo "Do another parallel test here"
    - echo "For example run a lint test"

build-job:   
  stage: build
  script:
    - "curl -X POST -F token=${CITOKEN} -F ref=main http://172.29.9.101/api/v4/projects/2/trigger/pipeline"
    - echo "Compile complete."
    - sleep 1


deploy:
  stage: deploy
  script:
    - echo "Do your deploy here"
  • 验证

此时,再新建一个分支,看是否会触发流水线的运行

image-20220513105612601

可以发现,此时新建分支不会触发流水线的运行了,符合预期:

image-20220513105640168

2.解决job运行重新下载代码

制品需要通过分布式缓存或者通过语法提前保存。

指的是,不同的runner节点。

通过下面的CI作业,观察第二个作业是否可以拿到第一个作业的文件;

variables:
  ENV_TYPE: "dev"

cibuild:
  tags:
    - build
  stage: build
  script:
    - ls -l 
    - echo 123 >test.yaml 
    - ls -l 

citest1:
  tags:
    - build
  stage: test
  script:
    - ls -l

分析: citest1作业并没有获取上个阶段产出文件, 因为默认每个作业运行时都会重新下载代码;重新下载代码会把其他文件删除掉

image-20230515070319296

如何解决?

  • 禁止后面的作业重复的下载代码;
  • 使用artifact关键字, 收集制品传递给其他作业;

image-20230515070558090

image-20230515070610628

因为作业可能被分配到不同的runner上运行的。

这里测试如下:

方法1:GIT_CHECKOUT

GIT_CHECKOUT变量,默认值为true,即作业每次运行都下载代码。按照我们目前的需求,需要禁止下载代码:

我们将此变量的值在全局配置为false,然后在第一个作业(.pre)中配置为true。也就实现了只在第一个作业下载代码而不会出现其他作业下载代码了。

GIT_CHECKOUT: "false" 

优化一下pipeline, 仅在pipelineInit作业中运行代码下载,其他作业跳过;

variables:
  ENV_TYPE: "dev"
  GIT_CHECKOUT: "false" 

pipelineInit:
  tags:
    - build
  stage: .pre
  variables:
    GIT_CHECKOUT: "true" 
  script:
    - ls -l 

cibuild:
  tags:
    - build
  stage: build
  script:
    - ls -l 
    - echo 123 >test.yaml 
    - ls -l 

citest1:
  tags:
    - build
  stage: test
  script:
    - ls -l 

查看citest1 作业中的输出, 发现工作目录中已经存在之前作业的产出文件;

image-20230515070940479

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-vjpTWDp4-1684397903089)(https://bucket-hg.oss-cn-shanghai.aliyuncs.com/img/image-20230515071027342.png)]

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-vHWKYVuX-1684397903089)(https://bucket-hg.oss-cn-shanghai.aliyuncs.com/img/image-20230515071131864.png)]

注意:

默认这里是git fetchgit shallow clobe值为20。

image-20230518075024939

但是在运行流水线时,报错:

image-20230518075718588

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-WCOv1Ox9-1684397903090)(https://bucket-hg.oss-cn-shanghai.aliyuncs.com/img/image-20230518075709985.png)]

image-20230518075658962

经过百度,说是要在项目-设置-ci/cd-General pipelines里设置为git clone才行。

image-20230518075841793

设置后,再次运行流水线:

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-Clg6AWIX-1684397903090)(https://bucket-hg.oss-cn-shanghai.aliyuncs.com/img/image-20230518075910471.png)]

此时,发现是ok的。

但是,在测试利用GIT_CHECKOUT下载代码时,就出现了问题,和老师的现象不一样

排查了好多问题,都解决不了……

报错如下:(在此步骤的上面)

最后,自己将项目-设置-ci/cd-General pipelines里设置为git fetch,然后把下面的git shadow clone设置为0后,再次运行流水线,就符合预期效果了。

image-20230518080154080

CI代码:

variables:
  ENV_TYPE: "dev"
  GIT_CHECKOUT: "false" 

pipelineInit:
  tags:
    - build
  stage: .pre
  variables:
    GIT_CHECKOUT: "true" 
  script:
    - ls -l 

cibuild:
  tags:
    - build
  stage: build
  script:
    - ls -l 
    - echo 123 >test.yaml 
    - ls -l 

citest1:
  tags:
    - build
  stage: test
  script:
    - ls -l 

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-jKEqPkAK-1684397903091)(https://bucket-hg.oss-cn-shanghai.aliyuncs.com/img/image-20230518080304481.png)]

image-20230518080314944

image-20230518080336773

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-6NxexaaB-1684397903091)(https://bucket-hg.oss-cn-shanghai.aliyuncs.com/img/image-20230518080400258.png)]

很奇怪,这是为什么呢??

⚠️ 汇总:

这个问题原因找到了。

之前项目里默认配置的是git fetchGit shallow clone=20,但是后续跑流水线时报错,百度了下,就改成git clone

但是后面在做GIT_CHECKOUT解决job运行重新下载代码时,测试现象和课程上的不一样。

最后改成了git fetchGit shallow clone=0,现象就一致了。

image-20230518075658962

image-20230518075024939

方法2:Artifacts

在作业结束时,收集作业中的文件或目录并以附件的形式保存在关联的作业中。作业运行完成后,制品将被存储到GitLab,并且可以在GitLab UI中下载。

  artifacts:
    name: "$CI_JOB_NAME-$CI_COMMIT_REF_NAME"
    when: on_success
    expire_in: '1 week'
    paths:
      - target/*.jar
  • name : 定义所创建的制品存档的名称;
  • when :定义何时进行制品收集;
  • expire_in : 制品的过期时间,过期自动清理;
  • paths: 定义要收集的制品文件或者目录信息;
  dependencies:
    - build   ## 作业名称

dependencies:

要获取哪些作业制品, 作业列表;只能是当前阶段之前的作业。如果空数组则跳过下载任何工件;不考虑先前作业的状态,因此,如果它失败或是未运行的手动作业,则不会发生错误。

基于artifacts和dependencies调整pipeline:

variables:
  ENV_TYPE: "dev"
  #GIT_CHECKOUT: "false" 

pipelineInit:
  tags:
    - build
  stage: .pre
  #variables:
    #GIT_CHECKOUT: "true" 
  script:
    - ls -l 

cibuild:
  tags:
    - build
  stage: build
  script:
    - ls -l 
    - echo 123 >test.yaml 
    - ls -l 
  artifacts:
    name: "$CI_JOB_NAME-$CI_COMMIT_REF_NAME"
    when: on_success
    expire_in: '1 week'
    paths:
      - test.yaml

citest1:
  tags:
    - build
  stage: test
  dependencies:
    - cibuild
  script:
    - ls -l 

查看结果:citest1作业会下载cibuild作业生成的制品;

image-20230515072525822

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-dSjSFaKB-1684397903092)(https://bucket-hg.oss-cn-shanghai.aliyuncs.com/img/image-20230515072553913.png)]

image-20230515072632575

以上问题已解决。

gitlab配置邮件通知反馈

  • 编辑/etc/gitlab/gitlab.rb文件开启gitlab email。这里以QQ邮箱为例
### GitLab email server settings
###! Docs: https://docs.gitlab.com/omnibus/settings/smtp.html
###! **Use smtp instead of sendmail/postfix.**

gitlab_rails['smtp_enable'] = true
gitlab_rails['smtp_address'] = "smtp.qq.com"
gitlab_rails['smtp_port'] = 465
gitlab_rails['smtp_user_name'] = "2560350642@qq.com"
gitlab_rails['smtp_password'] = "exnyvccnekcccjdgwecga"
gitlab_rails['smtp_domain'] = "smtp.qq.com"
gitlab_rails['smtp_authentication'] = "login"
gitlab_rails['smtp_enable_starttls_auto'] = true
gitlab_rails['smtp_tls'] = true

### Email Settings
gitlab_rails['gitlab_email_enabled'] = true
gitlab_rails['gitlab_email_from'] = '2560350642@qq.com'
gitlab_rails['gitlab_email_display_name'] = 'GitLab Admin'
  • 重新配置
gitlab-ctl stop ;
gitlab-ctl reconfigure ;
gitlab-ctl start gitlab-ctl status 
  • 登录gitlab-rails控制台,发送测试邮件:
su - git
gitlab-rails console

irb(main):002:0> Notify.test_email('2560350642@qq.com', 'test email', 'gitlab email test').deliver_now


Notify#test_email: processed outbound mail in 0.5ms
Delivered mail 5eba1b04de4e5_12903fe2ca0c79b0519ec@gitlab-995f97976-2nmb4.mail (1055.9ms)
Date: Tue, 12 May 2020 03:41:56 +0000
From: GitLab Admin <2560350642@qq.com>
Reply-To: GitLab Admin <noreply@192.168.1.200>
To: 2560350642@qq.com
Message-ID: <5eba1b04de4e5_12903fe2ca0c79b0519ec@gitlab-995f97976-2nmb4.mail>
Subject: Message Subject
Mime-Version: 1.0
Content-Type: text/html;
 charset=UTF-8
Content-Transfer-Encoding: 7bit
Auto-Submitted: auto-generated
X-Auto-Response-Suppress: All

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd">
<html><body><p>Message Body And Linuxea.com</p></body></html>

=> #<Mail::Message:70243016426420, Multipart: false, Headers: <Date: Tue, 12 May 2020 03:41:56 +0000>, <From: GitLab Admin <2560350642@qq.com>>, <Reply-To: GitLab Admin <noreply@192.168.1.200>>, <To: 2560350642@qq.com>, <Message-ID: <5eba1b04de4e5_12903fe2ca0c79b0519ec@gitlab-995f97976-2nmb4.mail>>, <Subject: Message Subject>, <Mime-Version: 1.0>, <Content-Type: text/html; charset=UTF-8>, <Content-Transfer-Encoding: 7bit>, <Auto-Submitted: auto-generated>, <X-Auto-Response-Suppress: All>>
  • 测试邮件:

image-20220513103454028

  • 配置gitlab流水线状态通知

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-OaNsizTe-1684397903093)(https://bucket-hg.oss-cn-shanghai.aliyuncs.com/img/image-20220513103526634.png)]

image-20220513103539983

image-20220513103552938

关于我

我的博客主旨:

  • 排版美观,语言精炼;
  • 文档即手册,步骤明细,拒绝埋坑,提供源码;
  • 本人实战文档都是亲测成功的,各位小伙伴在实际操作过程中如有什么疑问,可随时联系本人帮您解决问题,让我们一起进步!

🍀 语雀

https://www.yuque.com/xyy-onlyone

image-20230515221830698

🍀 csdn
https://blog.csdn.net/weixin_39246554?spm=1010.2135.3001.5421

image-20230107215149885

最后

好了,关于本次就到这里了,感谢大家阅读,最后祝大家生活快乐,每天都过的有意义哦,我们下期见!

image-20230516074158478

  • 0
    点赞
  • 5
    收藏
    觉得还不错? 一键收藏
  • 3
    评论
Gitlab CI/CD指的是Gitlab提供的持续集成和持续交付的功能。它可以帮助开发团队实现自动化的构建、测试和部署过程,从而提高开发效率和软件质量。 要使用GitLab CI/CD,需要熟悉.gitlab-ci.yml配置文件的语法及其属性。这个配置文件定义了构建和部署流程的步骤、依赖关系和环境变量等信息。你可以根据项目的需求自定义配置文件,GitLab CI/CD会根据配置文件的内容来执行相应的操作。 GitLab CI/CDGitLab中内置的一个功能强大的工具,它可以将连续集成、交付和部署应用于软件项目,而无需依赖第三方应用程序或集成。具体来说,它通过使用GitLab Runner来执行构建和部署作业,可以支持各种不同的项目类型和编程语言。你可以在GitLab的界面上配置、管理和监控CI/CD管道,查看运行结果和日志。 总之,GitLab CI/CD是一个强大的工具,可以帮助开发团队实现持续集成和持续交付,提高软件开发的效率和质量。<span class="em">1</span><span class="em">2</span><span class="em">3</span> #### 引用[.reference_title] - *1* [Gitlab CI/CD 简单介绍](https://blog.csdn.net/wangjiang_qianmo/article/details/122867335)[target="_blank" data-report-click={"spm":"1018.2226.3001.9630","extra":{"utm_source":"vip_chatgpt_common_search_pc_result","utm_medium":"distribute.pc_search_result.none-task-cask-2~all~insert_cask~default-1-null.142^v93^chatsearchT3_2"}}] [.reference_item style="max-width: 50%"] - *2* *3* [使用GitLab进行CI/CD简介](https://blog.csdn.net/FatTigerx/article/details/103766541)[target="_blank" data-report-click={"spm":"1018.2226.3001.9630","extra":{"utm_source":"vip_chatgpt_common_search_pc_result","utm_medium":"distribute.pc_search_result.none-task-cask-2~all~insert_cask~default-1-null.142^v93^chatsearchT3_2"}}] [.reference_item style="max-width: 50%"] [ .reference_list ]

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 3
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值