这里介绍下gitlab runner的docker in docker模式,即,gitlab runner安装在镜像里,cicd跑的服务器也在一个镜像里。
runner服务器上的操作
安装gitlab runner镜像和启动runner服务
docker run -d --name gitlab-runner --restart always \
-v /srv/gitlab-runner/config:/etc/gitlab-runner \
-v /var/run/docker.sock:/var/run/docker.sock \
gitlab/gitlab-runner:latest
注册一个runner
# url:gitlab 托管服务器地址,url和token可以从gitlab仓库页面/setting/cicd/runners页面获取
url=http://192.168...:8088/
token=DSVk4yPAZGUimsxd_b9Y
# cicd过程要在哪个镜像中跑?
img=pytorch:latest
# 这个runner要运行.gitlab-ci.yml中的哪些stage?如果stage的tags设置为ncnn_ci, 那么这个stage就在下面的runner中运行。
tag=ncnn_ci
# docker-pull-policy=never: 运行cicd时直接使用runner所在机器上已经有的镜像,不要重复拉去镜像,节省时间
# docker-gpus 'all'可以让启动的pytorch container有使用所有gpu的能力
# docker-shm-size 将虚拟内存设置为10G,避免训练出现资源不足问题
docker exec gitlab-runner gitlab-runner register -n \
--url $url \
--request-concurrency 6 \#可以同时运行多个job
--limit 6 \
--registration-token $token \
--tag-list $tag \
--executor docker \
--docker-image $img \
--docker-volumes /var/run/docker.sock:/var/run/docker.sock \
--docker-volumes ${HOME}:${HOME} \
--docker-privileged \
--docker-gpus 'all' \
--docker-shm-size 10000000 \
--docker-pull-policy never \
--description $tag
其他runner操作
- 取消注册所有runners
docker exec gitlab-runner gitlab-runner unregister --all-runners
# list all runners
docker exec gitlab-runner gitlab-runner list
# delete runer with name ncnn_ci
docker exec gitlab-runner gitlab-runner unregister -n ncnn_ci
gitlab操作
- 添加一个
.gitlab-ci.yml
文件,并提交到代码根目录,yml文件内容如下:
# 依次在image定义的镜像中运行下面三个stage
stages:
- package
- build
- deploy
my_package:
stage: package
script:
- echo "start package----------"
- /bin/bash tests/ci.sh
cache:
key: ${CI_PIPELINE_ID}
paths:
- target/
only:
# master有变动时触发
- master
tags:
# 定义这个stage的tag, 该stage会在对应runner上跑
- ncnn_ci
my_package2:
stage: package
script:
- echo "start package2----------"
my_build:
stage: build
cache:
key: ${CI_PIPELINE_ID}
paths:
- target/
script:
- echo "start build"
tags:
- ncnn_ci
my_deploy:
stage: deploy
script:
- echo "start deploy"
tags:
- ncnn_ci
注意:这里my_package和my_package2会并行运行。
验证runner
gitlab页面上cicd/pipeline上,选择run pipeline
。
参考资料:
https://www.zhihu.com/question/360019113
https://docs.gitlab.com/runner/commands/
https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runnersdocker-section
https://docs.gitlab.com/runner/executors/docker.html#how-pull-policies-work
https://docs.gitlab.com/ee/ci/docker/using_docker_images.html