本文需要掌握的技术
基础
- k8s
- helm charts
- gitlab runner
可选
- kubeseal
- sealed-secret
- S3
官方charts部署runner
添加repo
#国外
helm repo add gitlab https://charts.gitlab.io/
#国内
helm repo add gitlab-jh https://charts.gitlab.cn/
#查看helm库
helm repo list
确保能通过kubeconfig访问K8S集群
kubectl get pod -A
部署gitlab runner
#列出可用版本
helm search repo -l runner
#部署runner
helm install k8s-runner gitlab/gitlab-runner -n k8s-gitlab-runner --create-namespace
自定义chart部署runner
引用官方的charts,做自定义修改
初始化helm chart文件夹
#生成一个空chart
helm create chart-test
修改chart内容
apiVersion: v2
name: gitlab-runner
description: A Helm chart for Kubernetes
# A chart can be either an 'application' or a 'library' chart.
#
# Application charts are a collection of templates that can be packaged into versioned archives
# to be deployed.
#
# Library charts provide useful utilities or functions for the chart developer. They're included as
# a dependency of application charts to inject those utilities and functions into the rendering
# pipeline. Library charts do not define any templates and therefore cannot be deployed.
type: application
# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
# Versions are expected to follow Semantic Versioning (https://semver.org/)
version: 0.1.0
# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application. Versions are not expected to
# follow Semantic Versioning. They should reflect the version the application is using.
# It is recommended to use it with quotes.
appVersion: "1.16.0"
#对应runner的官方版本和名称以及仓库名
dependencies:
- name: gitlab-runner
repository: https://charts.gitlab.io/
version: 0.40.1
自定义templates
telmplates中的文件如果和官方charts里面对应目录文件名重合,那么会覆盖官方的,以自定义文件为准;
templates中可以新增一些部署,比如部署sealed-secret或者是configmap
apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
name: {{ .Release.Name }}-gitlab-runner
namespace: {{ default .Release.Namespace .Values.namespace | quote }}
labels:
app: runner
annotations:
"helm.sh/hook": pre-install,pre-upgrade
"helm.sh/weight": "1"
spec:
encryptedData:
runner-registration-token: {{ default "" .Values.runnerSealedRegistrationToken | quote }}
runner-token: {{ default "" .Values.runnersSealedToken | quote }}
template:
data: null
metadata:
name: {{ .Release.Name }}-gitlab-runner
namespace: {{ default .Release.Namespace .Values.namespace | quote }}
type: Opaque
配置Values
这里runnerSealedRegistrationToken和runnersSealedToken为kubeseal加密的字符串
对应的镜像换成私有仓库的地址会让pod启动更快:image和helper_image
runnerSealedRegistrationToken: "wefwefewfwefwef"
runnersSealedToken: "wefwefwefwefwef"
namespace: veevaorion-runner-large
gitlab-runner:
image: "repo/proxy/gitlab-runner:alpine-v14.10.1"
imagePullPolicy: IfNotPresent
unregisterRunners: true
terminationGracePeriodSeconds: 3600
concurrent: 10
checkInterval: 30
logLevel: debug
rbac:
create: true
rules:
- resources: ["configmaps", "pods", "pods/attach", "secrets", "services"]
apiGroups: [""]
verbs: ["get", "list", "watch", "create", "patch", "delete", "update"]
- apiGroups: [""]
resources: ["pods/exec"]
verbs: ["create", "patch", "delete"]
clusterWideAccess: false
podSecurityPolicy:
enabled: false
resourceNames:
- gitlab-runner
replicas: 1
gitlabUrl: https://gitlab.test.com/
nodeSelector:
usage: "runner-large"
runners:
tags: "veevaorion-group-large"
config: |
[[runners]]
[runners.kubernetes]
namespace = "{{.Release.Namespace}}"
image = "ubuntu:16.04"
pull_policy = ["if-not-present"]
helper_image = "repo/proxy/gitlab-runner-helper:x86_64-f761588f"
[runners.kubernetes.node_selector]
usage = "runner-large"
cache:
cacheType: s3
cachePath: "runner-cache"
cacheShared: true
s3ServerAddress: s3.amazonaws.com
s3BucketName: test-runner
s3BucketLocation: cn-northwest-1
s3CacheInsecure: false