一: 环境准备
|
1:系统环境 | # uname -a Linux node-16 3.10.0-1062.12.1.el7.x86_64 #1 SMP Tue Feb 4 23:02:59 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux # cat /etc/redhat-release CentOS Linux release 7.7.1908 (Core) # kubectl get nodes NAME STATUS ROLES AGE VERSION node-1 Ready master 49d v1.16.0 node-1 Ready worker 49d v1.16.0 node-1 Ready worker 49d v1.16.0 # # docker version Client: Docker Engine - Community Version: 18.09.6 API version: 1.39 Go version: go1.10.8 Git commit: 481bc77 Built: Sat May 4 02:33:34 2019 OS/Arch: linux/amd64 Experimental: false
Server: Docker Engine - Community Engine: Version: 18.09.6 API version: 1.39 (minimum version 1.12) Go version: go1.10.8 Git commit: 481bc77 Built: Sat May 4 02:41:08 2019 OS/Arch: linux/amd64 Experimental: false # docker images|grep jenkins jenkins/jenkins lts 5d1103b93f92 12 days ago 656MB #
|
| 2:下载源代码 | # git clone https://github.com/my-dlq/blog-example.git Cloning into 'blog-example'... remote: Enumerating objects: 752, done. remote: Counting objects: 100% (752/752), done. remote: Compressing objects: 100% (478/478), done. remote: Total 1954 (delta 285), reused 618 (delta 171), pack-reused 1202 Receiving objects: 100% (1954/1954), 390.83 KiB | 4.00 KiB/s, done. Resolving deltas: 100% (729/729), done. #
# pwd /root/software/blog-example/jenkins # ls jenkins-api-demo jenkins-ci&cd jenkins-deploy # |
二：存储pv/pvc准备
在 Kubenetes 环境下所起的应用都是一个个 Docker 镜像，为了保证应用重启的情况数据安全，所以需要将 Jenkins 持久化到存储中。这里用的是 NFS 网路存储，方便在 Kubernetes 环境下应用启动节点转义数据一致。当然也可以选择存储到本地，cent 等存储，但是为了保证应用数据一致，需要设置 Jenkins 固定到某一 Kubernetes 节点。
| 准备一台NFS服务器 | 先准备一台NFS服务器为K8S提供存储支持。 # yum install nfs-utils # # rpm -qa|grep nfs libnfsidmap-0.25-19.el7.x86_64 nfs-utils-1.3.0-0.65.el7.x86_64 # cat /etc/exports /ifs/kubernetes *(rw,sync,no_root_squash) /ifs/pv001 *(rw,no_root_squash) /ifs/pv002 *(rw,no_root_squash) /ifs/pv003 *(rw,no_root_squash) /ifs/pv004 *(rw,no_root_squash) /ifs/pv005 *(rw,no_root_squash) # systemctl start nfs # systemctl enable nfs |
| nfs-client-provisioner插件 | 并且要在每个Node上安装nfs-utils包，用于mount挂载时用。 由于K8S不支持NFS动态供给，还需要先安装上图中的nfs-client-provisioner插件： 这个插件主要的功能是为nfs 创建PV目 录和pv关链起来;
# cat deployment.yaml apiVersion: v1 kind: ServiceAccount metadata: name: nfs-client-provisioner --- kind: Deployment apiVersion: apps/v1 metadata: name: nfs-client-provisioner spec: replicas: 1 strategy: type: Recreate selector: matchLabels: app: nfs-client-provisioner template: metadata: labels: app: nfs-client-provisioner spec: serviceAccountName: nfs-client-provisioner containers: - name: nfs-client-provisioner image: quay.io/external_storage/nfs-client-provisioner:latest volumeMounts: - name: nfs-client-root mountPath: /persistentvolumes env: - name: PROVISIONER_NAME value: fuseim.pri/ifs - name: NFS_SERVER value: 100.100.100.16 - name: NFS_PATH value: /ifs/kubernetes volumes: - name: nfs-client-root nfs: server: 100.100.100.16 path: /ifs/kubernetes # cat class.yaml apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: managed-nfs-storage provisioner: fuseim.pri/ifs # or choose another name, must match deployment's env PROVISIONER_NAME' parameters: archiveOnDelete: "true" # kubectl apply -f . # kubectl get deployment|grep nfs-client-provisioner nfs-client-provisioner 1/1 1 1 42d # kubectl get sc|grep managed-nfs-storage managed-nfs-storage fuseim.pri/ifs 42d # kubectl get pods |grep nfs-client nfs-client-provisioner-657b74d847-9k7zz 1/1 Running 5 42d #
|
| 登录nfs pod 检查nfs挂载状态 | # kubectl exec -it nfs-client-provisioner-657b74d847-9k7zz sh / # / # df -Th Filesystem Type Size Used Available Use% Mounted on overlay overlay 48.9G 18.7G 30.2G 38% / tmpfs tmpfs 64.0M 0 64.0M 0% /dev tmpfs tmpfs 3.8G 0 3.8G 0% /sys/fs/cgroup 100.100.100.16:/ifs/kubernetes nfs4 48.9G 39.4G 9.5G 81% /persistentvolumes |
| 创建PV/PVC
| 创建 PV 绑定 NFS 创建的 Jenkins 目录，然后创建 PVC 绑定这个 PV，将此 PVC 用于后面创建 Jenkins 服务时挂载的存储。
# cat jenkins-pv-pvc.yaml apiVersion: v1 kind: PersistentVolume metadata: name: jenkins labels: app: jenkins spec: capacity: storage: 1Gi accessModes: - ReadWriteOnce persistentVolumeReclaimPolicy: Retain mountOptions: #NFS挂在选项 - hard - nfsvers=4.1 nfs: #NFS设置 path: /nfs/data/jenkins server: 100.100.100.16 --- kind: PersistentVolumeClaim apiVersion: v1 metadata: name: jenkins spec: # storageClassName: "managed-nfs-storage" accessModes: - ReadWriteOnce resources: requests: storage: 1Gi #生产环境空间一定要设置比较大点 selector: matchLabels: app: jenkins #
# kubectl apply -f jenkins-pv-pvc.yaml |
| 创建 ServiceAccount & ClusterRoleBinding | 此 kubernetes 集群用的是 RBAC 安全插件，必须创建权限给一个 ServiceAccount，然后将此 ServiceAccount 绑定到 Jenkins 服务，这样赋予 Jenkins 服务一定权限执行一些操作，为了方便，这里将 cluster-admin 绑定到 ServiceAccount 以保证 Jenkins 能拥有一定的权限。
注意：请提前修改 yaml 中的 namespace
# cat jenkins-rbac.yaml apiVersion: v1 kind: ServiceAccount metadata: name: jenkins-admin #ServiceAccount名 namespace: default #指定namespace，一定要修改成你自己的namespace labels: name: jenkins --- kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: jenkins-admin labels: name: jenkins subjects: - kind: ServiceAccount name: jenkins-admin namespace: default roleRef: kind: ClusterRole name: cluster-admin apiGroup: rbac.authorization.k8s.io # #
# kubectl create -f jenkins-rbac.yaml
|
三: jenkins部署
|
拉取jenkins源 | node1# docker pull jenkins/jenkins:lts node2# docker pull jenkins/jenkins:lts node3# docker pull jenkins/jenkins:lts
[root@node-1 ~]## docker images|grep jenkins jenkins/jenkins lts 5d1103b93f92 10 days ago 656MB # # docker images|grep jenkins jenkins/jenkins lts 5d1103b93f92 10 days ago 656MB [root@node-1 ~]#
[root@node-2 ~]# docker images|grep jenkins jenkins/jenkins lts 5d1103b93f92 10 days ago 656MB [root@node-3 ~]#
|
| 创建 Service & Deployment
| 这里开始部署 Jenkins 服务，创建 Service 与 Deployment，其中 Service 暴露两个接口 80880 与 50000。而 Deployment 里面要注意的是要设置上面创建的 ServiceAccount ，并且设置容器安全策略为“runAsUser: 0”以 Root 权限运行容器，而且暴露8080、50000两个端口。
创建 Service & Deployment 部署文件
# cat jenkins-deployment.yaml apiVersion: v1 kind: Service metadata: name: jenkins labels: app: jenkins spec: type: NodePort ports: - name: http port: 8080 #服务端口 targetPort: 8080 nodePort: 32001 #NodePort方式暴露 Jenkins 端口 - name: jnlp port: 50000 #代理端口 targetPort: 50000 nodePort: 32002 selector: app: jenkins --- apiVersion: apps/v1 kind: Deployment metadata: name: jenkins labels: app: jenkins spec: selector: matchLabels: app: jenkins replicas: 1 template: metadata: labels: app: jenkins spec: serviceAccountName: jenkins-admin containers: - name: jenkins image: jenkins/jenkins:lts securityContext: runAsUser: 0 #设置以ROOT用户运行容器 privileged: true #拥有特权 ports: - name: http containerPort: 8080 - name: jnlp containerPort: 50000 resources: limits: memory: 2Gi cpu: "1000m" requests: memory: 1Gi cpu: "500m" env: - name: LIMITS_MEMORY valueFrom: resourceFieldRef: resource: limits.memory divisor: 1Mi - name: "JAVA_TOOL_OPTIONS" value: " -Dfile.encoding=UTF-8 -Dsun.jnu.encoding=UTF-8 " - name: "JAVA_OPTS" #设置变量，指定时区和 jenkins slave 执行者设置 value: " -Xmx$(LIMITS_MEMORY)m -XshowSettings:vm -Dhudson.slaves.NodeProvisioner.initialDelay=0 -Dhudson.slaves.NodeProvisioner.MARGIN=50 -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85 -Duser.timezone=Asia/Shanghai " - name: "JENKINS_OPTS" value: "--prefix=/jenkins" #设置路径前缀加上 Jenkins volumeMounts: #设置要挂在的目录 - name: data mountPath: /var/jenkins_home volumes: - name: data persistentVolumeClaim: claimName: jenkins #设置PVC
# # kubectl apply -f jenkins-deployment.yaml service/jenkins created deployment.apps/jenkins created persistentvolumeclaim/jenkins created #
|
| 检查运行结果 | # kubectl get service|grep jenkins jenkins NodePort 10.0.0.130 <none> 8080:32001/TCP,50000:32002/TCP 2m24s # kubectl get deployment|grep jenkins jenkins 1/1 1 1 2m38s # kubectl get pod|grep jenkins jenkins-677485858c-gsv8q 1/1 Running 0 2m47s # kubectl get ep|grep jenkins jenkins 10.244.2.84:8080,10.244.2.84:50000 3m45s #
|
| 要查pvc/pv | # kubectl get pv|grep jenkins pvc-249c5e67-2711-4bb5-a625-39030efe77d1 2Gi RWO Delete Bound default/jenkins managed-nfs-storage 18s # kubectl get pvc|grep jenkins jenkins Bound pvc-249c5e67-2711-4bb5-a625-39030efe77d1 2Gi RWO managed-nfs-storage 28s # kubectl get sc NAME PROVISIONER AGE dynamic-cephfs ceph.com/cephfs 14d managed-nfs-storage fuseim.pri/ifs 45d nginx.storageclass ceph.com/rbd 47d # |
| 检查jenkins pod的日志 | # # kubectl logs jenkins-578dfd9b55-dqd58 -f 查看日志中生成的 Token 字符串 查看日志，默认给的token为：
************************************************************* ************************************************************* *************************************************************
Jenkins initial setup is required. An admin user has been created and a password generated. Please use the following password to proceed to installation:
c76db77497a843eeabedc8af565d2304
This may also be found at: /var/jenkins_home/secrets/initialAdminPassword
************************************************************* ************************************************************* *************************************************************
2020-06-08 07:46:57.877+0000 [id=39] INFO h.m.DownloadService$Downloadable#load: Obtained the updated data file for hudson.tasks.Maven.MavenInstaller 2020-06-08 07:46:57.877+0000 [id=39] INFO hudson.util.Retrier#start: Performed the action check updates server successfully at the attempt #1 2020-06-08 07:46:57.885+0000 [id=39] INFO hudson.model.AsyncPeriodicWork#lambda$doRun$0: Finished Download metadata. 148,705 ms 2020-06-08 07:47:01.066+0000 [id=26] INFO jenkins.InitReactorRunner$1#onAttained: Completed initialization 2020-06-08 07:47:01.080+0000 [id=19] INFO hudson.WebAppMain$3#run: Jenkins is fully up and running
^C [#
# kubectl get pod NAME READY STATUS RESTARTS AGE cephfs01-deployment-78b9d99fcf-pzghb 1/1 Running 0 13d jenkins-578dfd9b55-nbffb 1/1 Running 0 3m19s
参数说明： JAVA_TOOL_OPTIONS： 定义 JVM 环境变量，这里主要是设置编码参数，防止中文乱码 默认情况下，Jenkins生成代理是保守的。例如，如果队列中有两个构建，它不会立即生成两个执行器。它将生成一个执行器，并等待某个时间释放第一个执行器，然后再决定生成第二个执行器。Jenkins确保它生成的每个执行器都得到了最大限度的利用。如果你想覆盖这个行为，并生成一个执行器为每个构建队列立即不等待，所以在Jenkins启动时候添加这些参数: -Dhudson.slaves.NodeProvisioner.initialDelay=0 |
四: jenkins配置
| 启动安装jenkins 登录jenkins 管理界面 | http://100.100.100.16:32001/jenkins/
TOKEN 如下: c76db77497a843eeabedc8af565d2304
|
| 选择插件来安装 |
|
| 默认什么插件都不安装，后继需要再安装 |
|
| 创建一个管理员账号 |
|
| jenkins已就绪了 |
|
| Jenkins部署完成 |
# [# kubectl get pod,svc |grep jenkins pod/jenkins-677485858c-m2wng 1/1 Running 0 15m service/jenkins NodePort 10.0.0.112 <none> 8080:32001/TCP,50000:32002/TCP 15m #
|
| 增加ingress 解析 | [root@node-16 jenkins]# cat ingress.yaml apiVersion: v1 kind: List items: - apiVersion: extensions/v1beta1 kind: Ingress metadata: name: jenkins annotations: nginx.ingress.kubernetes.io/ssl-redirect: "true" nginx.ingress.kubernetes.io/proxy-body-size: 100m spec: rules: - host: jenkins.test.com http: paths: - path: backend: serviceName: jenkins servicePort: 8080 #
[# kubectl apply -f ingress.yaml
|
五: 安装Jenkins插件
| 选择manager Plugins |
|
| Jenkins 插件管理 |
Jenkins 默认的插件源是国外，下载时影响速度，需要将 插件源修改到国内的源！
直接在配置文件里面替换源！
|
| 修改实际下载的插件的地址 | 在持久化的存储目 录上面 # ls default-jenkins-pvc-b1b538b9-1e45-41a1-abdd-76b13ed4bf81
# cd default-jenkins-pvc-b1b538b9-1e45-41a1-abdd-76b13ed4bf81 [root@node-1 default-jenkins-pvc-b1b538b9-1e45-41a1-abdd-76b13ed4bf81]#
[root@node-1 default-jenkins-pvc-b1b538b9-1e45-41a1-abdd-76b13ed4bf81]# cd updates/ [root@node-1 updates]# ls default.json hudson.tasks.Maven.MavenInstaller [root@node-1 updates]#
修改default.json 文件 （下面的goolge服务器都在国外，访问，下载速度将很慢的 ,需要替换成国内的 源地址）
|
|
| 由于默认插件源在国外服务器，大多数网络无法顺利下载，需修改国内插件源地址，上面的default.json 编辑麻烦，使用下面的命令进行替换！
# pwd /ifs/kubernetes/default-jenkins-pvc-b1b538b9-1e45-41a1-abdd-76b13ed4bf81/updates [root@node-1 updates]# ls default.json hudson.tasks.Maven.MavenInstaller [root@node-1 updates]# [root@node-1 updates]# sed -i 's/http:\/\/updates.jenkins-ci.org\/download/https:\/\/mirrors.tuna.tsinghua.edu.cn\/jenkins/g' default.json && \ > sed -i 's/http:\/\/www.google.com/https:\/\/www.baidu.com/g' default.json [root@node-1 updates]# [root@node-1 updates]#
重启jenkins 生效！
http://IP:32001/jenkins/restart
|
| 安装pinline插件 |
|
| 创建一个job
显示有一个流水线，说明,pipeline 已成功安装 |
|
Jenkins 在kubernetes 环境部署完成！
扫一扫
1098
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
