在k8s中部署jenkins

在k8s中部署jenkins

1.在k8s中部署jenkins

image

注释: https://github.com/jenkinsci/kubernetes-plugin/tree/fc40c869edfd9e3904a9a56b0f80c5a25e988fa1/src/main/kubernetes

2. Jenkins在Kubernetes中动态创建代理

image

​ Jenkins Master/Slave架构

3. 案例,在k8s安装部署jenkins

3.1安装nfs共享磁盘
  • 所有node和磁盘共享服务器都安装nfs

    [root@jenkins-master ~]# yum install nfs-utils -y
    [root@k8s-node1 ~]# yum install nfs-utils -y
    [root@k8s-node2 ~]# yum install nfs-utils -y
    [root@k8s-node3 ~]# yum install nfs-utils -y
  • 配置共享磁盘

    [root@jenkins-master ~]# vim /etc/exports
    [root@jenkins-master ~]# cat /etc/exports
    /ifs/kubernetes *(rw,no_root_squash)
  • 创建共享磁盘目录

    [root@jenkins-master ~]# mkdir -p /ifs/kubernetes/
  • 每个node和磁盘共享都启动服务和设置开机启动

    # systemctl start nfs      #启动nfs
    # systemctl enable nfs     # 设置开机启动
    Created symlink from /etc/systemd/system/multi-user.target.wants/nfs-server.service to /usr/lib/systemd/system/nfs-server.service.
3.2 安装pvc动态扩容插件
3.2.1 下载插件
  • 克隆代码

    [root@k8s-master ~]# git clone https://github.com/kubernetes-incubator/external-storage
  • 我这里是先下在了安装包,通过sftp上传

    [root@k8s-master ~]# ll
    总用量 212
    -rw-------.  1 root root   1857 12月 15 09:32 anaconda-ks.cfg
    -rw-r--r--   1 root root 187757 12月 15 12:54 calico.yaml
    -rw-r--r--   1 root root   7591 12月 15 13:13 dashboard.yaml
    -rw-r--r--   1 root root   6615 12月 16 20:17 ingress_controller.yaml
    drwxr-xr-x   2 root root     64 12月 30 16:33 nfs-client
    -rw-r--r--.  1 root root   7867 12月 15 10:22 open.sh
    drw-r--r--  12 root root    144 12月 30 09:20 yaml
3.2.2 配置文件修改
  • 进入插件里

    [root@k8s-master ~]# cd nfs-client/
    [root@k8s-master nfs-client]# ll
    总用量 12
    -rw-r--r-- 1 root root  225 12月 30 16:33 class.yaml  # 创建存储类
    -rw-r--r-- 1 root root  994 12月 30 16:33 deployment.yaml  # 部署插件,需修改里面NFS服务器地址与共享目录
    -rw-r--r-- 1 root root 1526 12月 30 16:33 rbac.yaml    # 授权访问apiserver
  • 修改deployment配置文件

    [root@k8s-master nfs-client]# vim deployment.yaml 
    [root@k8s-master nfs-client]# cat deployment.yaml 
    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: nfs-client-provisioner
    ---
    kind: Deployment
    apiVersion: apps/v1 
    metadata:
      name: nfs-client-provisioner
    spec:
      replicas: 1
      strategy:
        type: Recreate
      selector:
        matchLabels:
          app: nfs-client-provisioner
      template:
        metadata:
          labels:
            app: nfs-client-provisioner
        spec:
          serviceAccountName: nfs-client-provisioner
          containers:
            - name: nfs-client-provisioner
              # 把国外源修改为国内源
              #image: quay.io/external_storage/nfs-client-provisioner:latest
              image: shichao01/nfs-client-provisioner:latest
              volumeMounts:
                - name: nfs-client-root
                  mountPath: /persistentvolumes
              env:
                - name: PROVISIONER_NAME
                  value: fuseim.pri/ifs
                - name: NFS_SERVER
                  # 修改nfs连接地址
                  #value: 192.168.1.61
                  value: 10.100.24.85 
                - name: NFS_PATH
                  value: /ifs/kubernetes    #共享路径,根据情况修改
          volumes:
            - name: nfs-client-root
              nfs:
                server: 10.100.24.85 
                path: /ifs/kubernetes
  • 修改class文件

    [root@k8s-master nfs-client]# vim class.yaml 
    [root@k8s-master nfs-client]# cat class.yaml 
    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
      name: managed-nfs-storage
    provisioner: fuseim.pri/ifs # or choose another name, must match deployment's env PROVISIONER_NAME'
    parameters:
      archiveOnDelete: "true"
  • 配置rbac.yaml文件

    [root@k8s-master nfs-client]# vim rbac.yaml 
    [root@k8s-master nfs-client]# cat rbac.yaml 
    kind: ServiceAccount
    apiVersion: v1
    metadata:
      name: nfs-client-provisioner
    ---
    kind: ClusterRole
    apiVersion: rbac.authorization.k8s.io/v1
    metadata:
      name: nfs-client-provisioner-runner
    rules:
      - apiGroups: [""]
        resources: ["persistentvolumes"]
        verbs: ["get", "list", "watch", "create", "delete"]
      - apiGroups: [""]
        resources: ["persistentvolumeclaims"]
        verbs: ["get", "list", "watch", "update"]
      - apiGroups: ["storage.k8s.io"]
        resources: ["storageclasses"]
        verbs: ["get", "list", "watch"]
      - apiGroups: [""]
        resources: ["events"]
        verbs: ["create", "update", "patch"]
    ---
    kind: ClusterRoleBinding
    apiVersion: rbac.authorization.k8s.io/v1
    metadata:
      name: run-nfs-client-provisioner
    subjects:
      - kind: ServiceAccount
        name: nfs-client-provisioner
        namespace: default
    roleRef:
      kind: ClusterRole
      name: nfs-client-provisioner-runner
      apiGroup: rbac.authorization.k8s.io
    ---
    kind: Role
    apiVersion: rbac.authorization.k8s.io/v1
    metadata:
      name: leader-locking-nfs-client-provisioner
    rules:
      - apiGroups: [""]
        resources: ["endpoints"]
        verbs: ["get", "list", "watch", "create", "update", "patch"]
    ---
    kind: RoleBinding
    apiVersion: rbac.authorization.k8s.io/v1
    metadata:
      name: leader-locking-nfs-client-provisioner
    subjects:
      - kind: ServiceAccount
        name: nfs-client-provisioner
        # replace with namespace where provisioner is deployed
        namespace: default
    roleRef:
      kind: Role
      name: leader-locking-nfs-client-provisioner
      apiGroup: rbac.authorization.k8s.io
3.2.3 启动所有服务
[root@k8s-master nfs-client]# kubectl apply -f .
storageclass.storage.k8s.io/managed-nfs-storage created
serviceaccount/nfs-client-provisioner created
deployment.apps/nfs-client-provisioner created
serviceaccount/nfs-client-provisioner unchanged
clusterrole.rbac.authorization.k8s.io/nfs-client-provisioner-runner created
clusterrolebinding.rbac.authorization.k8s.io/run-nfs-client-provisioner created
role.rbac.authorization.k8s.io/leader-locking-nfs-client-provisioner created
rolebinding.rbac.authorization.k8s.io/leader-locking-nfs-client-provisioner created
3.2.4 查看服务是否启动
[root@k8s-master nfs-client]# kubectl get pods
NAME                                   READY   STATUS    RESTARTS   AGE
configmap-demo-pod                     1/1     Running   0          4d1h
my-hostpath                            1/1     Running   4          2d1h
my-pod                                 1/1     Running   0          7h35m
nfs-client-provisioner-95c9579-m2f69   1/1     Running   0          52s
secret-demo-pod                        1/1     Running   0          3d18h
web-nfs-84f8d7bf8d-6mj75               1/1     Running   0          44h
web-nfs-84f8d7bf8d-n4tpk               1/1     Running   0          44h
web-nfs-84f8d7bf8d-qvd2z               1/1     Running   0          44h
3.2.5 验证服务
[root@k8s-master nfs-client]# kubectl get sc
NAME                  PROVISIONER      RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
managed-nfs-storage   fuseim.pri/ifs   Delete          Immediate           false                  3m46s
3.3 部署jenkins
  • 创建jenkins的yaml配置文件路径

    [root@k8s-master yaml]# mkdir -p jenkins-master
    [root@k8s-master yaml]# cd jenkins-master/
  • 创建yaml的配置文件

    • 编写statefulSet的jenkins配置文件

      [root@k8s-master jenkins-master]# vim jenkins-statefulset.yaml 
      [root@k8s-master jenkins-master]# cat jenkins-statefulset.yaml 
      apiVersion: apps/v1
      kind: StatefulSet
      metadata:
        name: jenkins
        labels:
          name: jenkins
      spec:
        serviceName: "jenkins"
        replicas: 1
        updateStrategy:
          type: RollingUpdate
        selector:
          matchLabels:
            name: jenkins
        template:
          metadata:  
            name: jenkins
            labels:
              name: jenkins
          spec:
            terminationGracePeriodSeconds: 10
            serviceAccountName: jenkins
            containers:
              - name: jenkins
                image: jenkins/jenkins:lts-alpine
                imagePullPolicy: Always
                ports:
                  - containerPort: 8080
                  - containerPort: 50000
                resources:
                  limits:
                    cpu: 1
                    memory: 1Gi
                  requests:
                    cpu: 0.5
                    memory: 500Mi
                env:
                  - name: LIMITS_MEMORY
                    valueFrom:
                      resourceFieldRef:
                        resource: limits.memory
                        divisor: 1Mi
                  - name: JAVA_OPTS
                    value: -Xmx$(LIMITS_MEMORY)m -XshowSettings:vm -Dhudson.slaves.NodeProvisioner.initialDelay=0 -Dhudson.slaves.NodeProvisioner.MARGIN=50 -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85
                volumeMounts:
                  - name: jenkins-home
                    mountPath: /var/jenkins_home
                livenessProbe:
                  httpGet:
                    path: /login
                    port: 8080
                  initialDelaySeconds: 60
                  timeoutSeconds: 5
                  failureThreshold: 12
                readinessProbe:
                  httpGet:
                    path: /login
                    port: 8080
                  initialDelaySeconds: 60
                  timeoutSeconds: 5
                  failureThreshold: 12
            securityContext:
              fsGroup: 1000
        volumeClaimTemplates:
        - metadata:
            name: jenkins-home
          spec:
            storageClassName: "managed-nfs-storage"
            accessModes: [ "ReadWriteOnce" ]
            resources:
              requests:
                storage: 1Gi
    • 编写service服务配置文件

      [root@k8s-master jenkins-master]# vim jenkins-service.yaml 
      [root@k8s-master jenkins-master]# cat jenkins-service.yaml 
      apiVersion: v1
      kind: Service
      metadata:
        name: jenkins
        namespace: default
      spec:
        selector:
          name: jenkins
        type: NodePort
        ports:
          -
            name: http
            port: 80
            targetPort: 8080
            protocol: TCP
            nodePort: 30006
          -
            name: agent
            port: 50000
            protocol: TCP
    • 编写ingress对外服务提供配置文件

      [root@k8s-master jenkins-master]# vi jenkins-ingress.yaml
      [root@k8s-master jenkins-master]# cat jenkins-ingress.yaml
      apiVersion: extensions/v1beta1
      kind: Ingress
      metadata:
        name: jenkins
        annotations:
          nginx.ingress.kubernetes.io/ssl-redirect: "true"
          kubernetes.io/tls-acme: "true"
          # 如果上传插件超出默认会报"413 Request Entity Too Large", 增加 client_max_body_size
          nginx.ingress.kubernetes.io/proxy-body-size: 50m
          nginx.ingress.kubernetes.io/proxy-request-buffering: "off"
          # nginx-ingress controller版本小于 0.9.0.beta-18 的配置
          ingress.kubernetes.io/ssl-redirect: "true"
          ingress.kubernetes.io/proxy-body-size: 50m
          ingress.kubernetes.io/proxy-request-buffering: "off"
      spec:
        rules:
        - host: jenkins.scajy.org
          http:
            paths:
            - path: /
              backend:
                serviceName: jenkins
                servicePort: 80
    • 配置jenkins的rbac可以访问api资源

      [root@k8s-master jenkins-master]# vim jenkins-rbac.yaml
      [root@k8s-master jenkins-master]# cat jenkins-rbac.yaml
      ---
      # 创建名为jenkins的ServiceAccount
      apiVersion: v1
      kind: ServiceAccount
      metadata:
        name: jenkins
      
      ---
      # 创建名为jenkins的Role,授予允许管理API组的资源Pod
      kind: Role
      apiVersion: rbac.authorization.k8s.io/v1
      metadata:
        name: jenkins
      rules:
      - apiGroups: [""]
        resources: ["pods"]
        verbs: ["create","delete","get","list","patch","update","watch"]
      - apiGroups: [""]
        resources: ["pods/exec"]
        verbs: ["create","delete","get","list","patch","update","watch"]
      - apiGroups: [""]
        resources: ["pods/log"]
        verbs: ["get","list","watch"]
      - apiGroups: [""]
        resources: ["secrets"]
        verbs: ["get"]
      
      ---
      # 将名为jenkins的Role绑定到名为jenkins的ServiceAccount
      apiVersion: rbac.authorization.k8s.io/v1
      kind: RoleBinding
      metadata:
        name: jenkins
      roleRef:
        apiGroup: rbac.authorization.k8s.io
        kind: Role
        name: jenkins
      subjects:
      - kind: ServiceAccount
        name: jenkins
    • 配置jenkins的account的rbac权限

      [root@k8s-master jenkins-master]# vim jenkins-account.yaml 
      [root@k8s-master jenkins-master]# cat jenkins-account.yaml 
      # In GKE need to get RBAC permissions first with
      # kubectl create clusterrolebinding cluster-admin-binding --clusterrole=cluster-admin [--user=<user-name>|--group=<group-name>]
      
      ---
      apiVersion: v1
      kind: ServiceAccount
      metadata:
        name: jenkins
      
      ---
      kind: Role
      apiVersion: rbac.authorization.k8s.io/v1
      metadata:
        name: jenkins
      rules:
      - apiGroups: [""]
        resources: ["pods"]
        verbs: ["create","delete","get","list","patch","update","watch"]
      - apiGroups: [""]
        resources: ["pods/exec"]
        verbs: ["create","delete","get","list","patch","update","watch"]
      - apiGroups: [""]
        resources: ["pods/log"]
        verbs: ["get","list","watch"]
      - apiGroups: [""]
        resources: ["secrets"]
        verbs: ["get"]
      
      ---
      apiVersion: rbac.authorization.k8s.io/v1
      kind: RoleBinding
      metadata:
        name: jenkins
      roleRef:
        apiGroup: rbac.authorization.k8s.io
        kind: Role
        name: jenkins
      subjects:
      - kind: ServiceAccount
        name: jenkins
    • 启动所有配置文件

      [root@k8s-master jenkins]# kubectl apply -f .
      Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
      ingress.extensions/jenkins created
      serviceaccount/jenkins created
      Warning: rbac.authorization.k8s.io/v1beta1 Role is deprecated in v1.17+, unavailable in v1.22+; use rbac.authorization.k8s.io/v1 Role
      role.rbac.authorization.k8s.io/jenkins created
      Warning: rbac.authorization.k8s.io/v1beta1 RoleBinding is deprecated in v1.17+, unavailable in v1.22+; use rbac.authorization.k8s.io/v1 RoleBinding
      rolebinding.rbac.authorization.k8s.io/jenkins created
      serviceaccount/jenkins unchanged
      role.rbac.authorization.k8s.io/jenkins unchanged
      rolebinding.rbac.authorization.k8s.io/jenkins unchanged
      service/jenkins created
      statefulset.apps/jenkins created
3.4 检查服务是否启动
[root@k8s-master jenkins-master]# kubectl get pods,service -o wide
NAME                                          READY   STATUS    RESTARTS   AGE    IP               NODE        NOMINATED NODE   READINESS GATES
pod/jenkins-0                                 1/1     Running   8          127m   10.244.36.72     k8s-node1   <none>           <none>
pod/nfs-client-provisioner-5d99f8d7b9-562ff   1/1     Running   1          12d    10.244.169.130   k8s-node2   <none>           <none>
pod/nginx-6799fc88d8-9fchc                    1/1     Running   0          12d    10.244.169.131   k8s-node2   <none>           <none>

NAME                 TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                        AGE    SELECTOR
service/jenkins      NodePort    10.107.141.118   <none>        80:30006/TCP,50000:32291/TCP   163m   name=jenkins
service/kubernetes   ClusterIP   10.96.0.1        <none>        443/TCP                        13d    <none>
service/nginx        NodePort    10.99.236.10     <none>        80:30939/TCP                   12d    app=nginx
[root@k8s-master jenkins-master]# kubectl get ep
NAME             ENDPOINTS                              AGE
fuseim.pri-ifs   <none>                                 12d
jenkins          10.244.36.72:8080,10.244.36.72:50000   164m
kubernetes       10.100.24.81:6443                      13d
nginx            10.244.169.131:80                      12d
3.5 使用浏览器访问测试
  • 这里先绑定ingress的hosts地址image

  • 使用域名访问测试image

  • 使用IP访问测试image

  • 注释:

    • 到这里,我们发现都可以访问
3.6 浏览器访问进行jenkins安装

image

  • 使用命令查看密码

    [root@k8s-master jenkins-master]# kubectl exec -it jenkins-0 cat /var/jenkins_home/secrets/initialAdminPassword
    kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
    8848ef969b454f849121e5790ea7dd52

imageimage

  • 跳过插件安装image

imageimageimage

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值