Kubernetes和Jenkins——基于Kubernetes构建Jenkins持续集成平台

最新推荐文章于 2023-07-06 14:39:44 发布

置顶 stan Z

最新推荐文章于 2023-07-06 14:39:44 发布

阅读量1w

点赞数 15

分类专栏： Kubernetes CI/CD 文章标签： kubernetes jenkins docker java ci/cd

本文链接：https://blog.csdn.net/Cantevenl/article/details/116722510

版权

CI/CD 同时被 2 个专栏收录

7 篇文章 0 订阅

订阅专栏

Kubernetes

2 篇文章 0 订阅

订阅专栏

Kubernetes+Docker+Jenkins持续集成架构图

在这里插入图片描述

构建K8S集群
Jenkins调度K8S API
动态生成 Jenkins Slave pod
Slave pod 拉取 Git 代码／编译／打包镜像
推送到镜像仓库 Harbor
Slave 工作完成，Pod 自动销毁
部署到测试或生产 Kubernetes平台

Kubernetes+Docker+Jenkins持续集成方案好处

服务高可用
- 当 Jenkins Master 出现故障时，Kubernetes 会自动创建一个新的 Jenkins Master容器，并且将 Volume 分配给新创建的容器，保证数据不丢失，从而达到集群服务高可用
动态伸缩，合理使用资源
- 每次运行 Job 时，会自动创建一个 Jenkins Slave，Job 完成后，Slave自动注销并删除容器，资源自动释放
- 而且 Kubernetes 会根据每个资源的使用情况，动态分配Slave 到空闲的节点上创建，降低出现因某节点资源利用率高，还排队等待在该节点的情况。
扩展性好
当 Kubernetes 集群的资源严重不足而导致 Job 排队等待时，可以很容易的添加一个Kubernetes Node 到集群中，从而实现扩展。

Kubeadm安装Kubernetes

K8S详细可以参考：Kubernetes

Kubernetes的架构

在这里插入图片描述

API Server：用于暴露Kubernetes API，任何资源的请求的调用操作都是通过kube-apiserver提供的接口进行的。
Etcd：是Kubernetes提供默认的存储系统，保存所有集群数据，使用时需要为etcd数据提供备份计划。
Controller-Manager：作为集群内部的管理控制中心，负责集群内的Node、Pod副本、服务端点（Endpoint）、命名空间（Namespace）、服务账号（ServiceAccount）、资源定额（ResourceQuota）的管理，当某个Node意外宕机时，Controller Manager会及时发现并执行自动化修复流程，确保集群始终处于预期的工作状态。
Scheduler：监视新创建没有分配到Node的Pod，为Pod选择一个Node。
Kubelet：负责维护容器的生命周期，同时负责Volume和网络的管理
Kube proxy：是Kubernetes的核心组件，部署在每个Node节点上，它是实现Kubernetes Service的通信与负载均衡机制的重要组件

安装环境准备

主机	ip	安装的软件
k8s-master	192.168.188.116	kube-apiserver、kube-controller-manager、kube-scheduler、docker、etcd、calico，NFS
k8s-node1	192.168.188.117	kubelet、kubeproxy、Docker
k8s-node2	192.168.188.118	kubelet、kubeproxy、Docker
harbor服务器	192.168.188.119	Harbor
gitlab服务器	192.168.188.120	Gitlab

三台k8s服务器都需要完成

关闭防火墙
systemctl stop firewalld
systemctl disable firewalld

关闭selinux
# 临时关闭
setenforce 0
# 永久关闭
sed -i 's/enforcing/disabled/' /etc/selinux/config  
  
关闭swap
# 临时
swapoff -a 
# 永久关闭
sed -ri 's/.*swap.*/#&/' /etc/fstab

# 根据规划设置主机名【master节点上操作】
hostnamectl set-hostname master
# 根据规划设置主机名【node1节点操作】
hostnamectl set-hostname node1
# 根据规划设置主机名【node2节点操作】
hostnamectl set-hostname node2

添加ip到hosts
cat >> /etc/hosts << EOF
192.168.188.116 master
192.168.188.117 node1
192.168.188.118 node2
EOF

设置系统参数
设置允许路由转发，不对bridge的数据进行处理
cat > /etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
vm.swappiness = 0
EOF

执行文件
sysctl -p /etc/sysctl.d/k8s.conf

kube-proxy开启ipvs的前置条件

cat > /etc/sysconfig/modules/ipvs.modules <<EOF 
#!/bin/bash 
modprobe -- ip_vs 
modprobe -- ip_vs_rr 
modprobe -- ip_vs_wrr 
modprobe -- ip_vs_sh 
modprobe -- nf_conntrack_ipv4 
EOF

chmod 755 /etc/sysconfig/modules/ipvs.modules && bash 

/etc/sysconfig/modules/ipvs.modules && lsmod | grep -e ip_vs -e nf_conntrack_ipv4

安装Docker、kubelet、kubeadm、kubectl

所有节点安装Docker、kubeadm、kubelet，Kubernetes默认CRI（容器运行时）为Docker，因此先安装Docker

安装Docker

首先配置一下Docker的阿里yum源
安装需要的安装包
yum install -y yum-utils

设置镜像仓库
我们用阿里云

yum-config-manager \
    --add-repo \
    http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

更新yum软件包索引
yum makecache fast

安装docker	docker-ce 社区
yum -y install docker-ce

查看版本
docker version

设置开机启动
systemctl enable docker --now

配置docker的镜像源
mkdir -p /etc/docker

这个是我自己阿里云的加速 每个人都不一样 可以去阿里云官方查看

tee /etc/docker/daemon.json <<-EOF
 {
   "registry-mirrors": ["https://m0rsqupc.mirror.aliyuncs.com"]
 }
EOF

验证
[root@master ~]# cat /etc/docker/daemon.json 
 {
   "registry-mirrors": ["https://m0rsqupc.mirror.aliyuncs.com"]
 }

然后重启docker
systemctl restart docker

安装kubelet、kubeadm、kubectl

kubeadm: 用来初始化集群的指令
kubelet: 在集群中的每个节点上用来启动 pod 和 container 等
kubectl: 用来与集群通信的命令行工具

添加kubernetes软件源

cat > /etc/yum.repos.d/kubernetes.repo << EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

安装kubelet、kubeadm、kubectl，同时指定版本

yum install -y kubelet-1.18.0 kubeadm-1.18.0 kubectl-1.18.0

systemctl enable kubelet --now

部署Kubernetes Master（master节点）

kubeadm init --kubernetes-version=1.18.0 --apiserver-advertise-address=192.168.188.116 --image-repository registry.aliyuncs.com/google_containers --kubernetes-version v1.18.0 --service-cidr=10.96.0.0/12  --pod-network-cidr=10.244.0.0/16

由于默认拉取镜像地址k8s.gcr.io国内无法访问，这里指定阿里云镜像仓库地址，（执行上述命令会比较慢，因为后台其实已经在拉取镜像了）
我们 docker images 命令即可查看已经拉取的镜像

在这里插入图片描述

表示kubernetes的镜像已经安装成功
红色圈出来的部分是下面加入从节点需要使用的命令

kubeadm join 192.168.188.116:6443 --token ic49lg.zuwab84r0zfs6bbr \
    --discovery-token-ca-cert-hash sha256:270285cba2080b1e291a3a2b3b21730616b59c95c55ca6f950fecf7b68869b97

使用kubectl工具
mkdir -p $HOME/.kube

cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

chown $(id -u):$(id -g) $HOME/.kube/config

查看运行的节点
kubectl get nodes

目前有一个master节点已经运行了，但是还处于未准备状态

安装Calico
Calico是一个网络组件，作用是实现master和子节点实现网络通讯功能

mkdir k8s 
cd k8s 

wget https://docs.projectcalico.org/v3.10/getting-started/kubernetes/installation/hosted/kubernetes-datastore/calico-networking/1.7/calico.yaml

sed -i 's/192.168.0.0/10.244.0.0/g' calico.yaml 

安装calico的组件
kubectl apply -f calico.yaml

查看pod
kubectl get pod --all-namespaces
必须是全部READY 状态必须是Running

查看更详细的信息
kubectl get pod --all-namespaces -o wide

在这里插入图片描述

加入Kubernetes Node（Slave节点）

需要到 node1 和 node2服务器，向集群添加新节点

执行在kubeadm init输出的kubeadm join命令

以下的命令是在master初始化完成后，每个人的都不一样！！！需要复制自己生成的

kubeadm join 192.168.188.116:6443 --token ic49lg.zuwab84r0zfs6bbr \
    --discovery-token-ca-cert-hash sha256:270285cba2080b1e291a3a2b3b21730616b59c95c55ca6f950fecf7b68869b97

查看kubelet是否开启
systemctl status kubelet

然后查看node信息
kubectl get nodes

[root@master k8s]# kubectl get nodes
NAME     STATUS   ROLES    AGE     VERSION
master   Ready    master   17m     v1.18.0
node1    Ready    <none>   2m27s   v1.18.0
node2    Ready    <none>   2m25s   v1.18.0

如果Status全部为Ready，代表集群环境搭建成功

kubectl常用命令

kubectl get nodes 查看所有主从节点的状态 
kubectl get ns 获取所有namespace资源 
kubectl get pods -n {$nameSpace} 获取指定namespace的pod 
kubectl describe pod的名称 -n {$nameSpace} 查看某个pod的执行过程 
kubectl logs --tail=1000 pod的名称 | less 查看日志 
kubectl create -f xxx.yml 通过配置文件创建一个集群资源对象 
kubectl delete -f xxx.yml 通过配置文件删除一个集群资源对象 
kubectl delete pod名称 -n {$nameSpace} 通过pod删除集群资源 
kubectl get service -n {$nameSpace} 查看pod的service情况

基于Kubernetes构建Jenkins持续集成平台

在K8S的基础上创建一个Jenkins主节点
在K8S上再创建Jenkins的从节点，来帮助我们进行项目的构建

安装和配置NFS

NFS（Network File System），它最大的功能就是可以通过网络，让不同的机器、不同的操作系统可以共享彼此的文件。我们可以利用NFS共享Jenkins运行的配置文件、Maven的仓库依赖文件等

我们把NFS服务器安装在K8S主节点上

安装NFS服务（这个需要在所有K8S的节点上安装）
yum install -y nfs-utils

创建共享目录（这个只需要在master节点）
mkdir -p /opt/nfs/jenkins
编写NFS的共享配置
vim /etc/exports 

/opt/nfs/jenkins	*(rw,no_root_squash) 

*代表对所有IP都开放此目录，rw是读写，no_root_squash不压制root权限

启动服务
systemctl enable nfs --now

查看NFS共享目录
showmount -e 192.168.188.116

[root@node1 ~]# showmount -e 192.168.188.116
Export list for 192.168.188.116:
/opt/nfs/jenkins *

在Kubernetes安装Jenkins-Master

创建NFS client provisioner

nfs-client-provisioner 是一个Kubernetes的简易NFS的外部provisioner，本身不提供NFS，需要现有的NFS服务器提供存储

需要编写yaml了

StorageClass.yaml
持久化存储Storageclass
kubectl explain StorageClass查看kind和version
定义名称为managed-nfs-storage

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: managed-nfs-storage
provisioner: fuseim.pri/ifs # or choose another name, must match deployment's env PROVISIONER_NAME'
parameters:
  archiveOnDelete: "true"

deployment.yaml
部署NFS client provisioner

apiVersion: storage.k8s.io/v1
kind: ServiceAccount
metadata:
  name: nfs-client-provisioner
---
kind: Deployment
apiVersion: apps/v1
metadata:
  name: nfs-client-provisioner
spec:
  replicas: 1
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: nfs-client-provisioner
  template:
    metadata:
      labels:
        app: nfs-client-provisioner
    spec:
      serviceAccountName: nfs-client-provisioner
      containers:
        - name: nfs-client-provisioner
          image: lizhenliang/nfs-client-provisioner:latest
          volumeMounts:
            - name: nfs-client-root
              mountPath: /persistentvolumes
          env:
            - name: PROVISIONER_NAME
              value: fuseim.pri/ifs
            - name: NFS_SERVER
              value: 192.168.188.116
            - name: NFS_PATH
              value: /opt/nfs/jenkins/
      volumes:
        - name: nfs-client-root
          nfs:
            server: 192.168.188.116
            path: /opt/nfs/jenkins/

最后是一个权限文件rbac.yaml

kind: ServiceAccount
apiVersion: v1
metadata:
  name: nfs-client-provisioner
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nfs-client-provisioner-runner
rules:
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: run-nfs-client-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    namespace: default
roleRef:
  kind: ClusterRole
  name: nfs-client-provisioner-runner
  apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
rules:
  - apiGroups: [""]
    resources: ["endpoints"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: default
roleRef:
  kind: Role
  name: leader-locking-nfs-client-provisioner
  apiGroup: rbac.authorization.k8s.io

构建pod

将三个文件都写在同一个目录里面
kubectl create -f .

[root@master nfs-client]# kubectl create -f .
storageclass.storage.k8s.io/managed-nfs-storage created
serviceaccount/nfs-client-provisioner created
deployment.apps/nfs-client-provisioner created
clusterrole.rbac.authorization.k8s.io/nfs-client-provisioner-runner created
clusterrolebinding.rbac.authorization.k8s.io/run-nfs-client-provisioner created
role.rbac.authorization.k8s.io/leader-locking-nfs-client-provisioner created
rolebinding.rbac.authorization.k8s.io/leader-locking-nfs-client-provisioner created
Error from server (AlreadyExists): error when creating "rbac.yaml": serviceaccounts "nfs-client-provisioner" already exists

查看pod
kubectl get pod

[root@master nfs-client]# kubectl get pod
NAME                                      READY   STATUS    RESTARTS   AGE
nfs-client-provisioner-795b4df87d-zchrq   1/1     Running   0          2m4s

安装Jenkins-Master

还是需要我们自己写Jenkins的Yaml文件

rbac.yaml
和权限有关的信息，把jenkins主节点的一些权限加入到k8s的管理下

kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: jenkins
  namespace: kube-ops
rules:
  - apiGroups: ["extensions", "apps"]
    resources: ["deployments"]
    verbs: ["create", "delete", "get", "list", "watch", "patch", "update"]
  - apiGroups: [""]
    resources: ["services"]
    verbs: ["create", "delete", "get", "list", "watch", "patch", "update"]
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["create","delete","get","list","patch","update","watch"]
  - apiGroups: [""]
    resources: ["pods/exec"]
    verbs: ["create","delete","get","list","patch","update","watch"]
  - apiGroups: [""]
    resources: ["pods/log"]
    verbs: ["get","list","watch"]
  - apiGroups: [""]
    resources: ["secrets"]
    verbs: ["get"]

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: jenkins
  namespace: kube-ops
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: jenkins
subjects:
  - kind: ServiceAccount
    name: jenkins
    namespace: kube-ops
    
---

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: jenkinsClusterRole
  namespace: kube-ops
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
  resources: ["pods/exec"]
  verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
  resources: ["pods/log"]
  verbs: ["get","list","watch"]
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get"]
 
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: jenkinsClusterRuleBinding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: jenkinsClusterRole
subjects:
- kind: ServiceAccount
  name: jenkins
  namespace: kube-ops

ServiceaAcount.yaml
jenkins主节点的一些ServiceaAcount信息

apiVersion: v1
kind: ServiceAccount
metadata:
  name: jenkins
  namespace: kube-ops

StatefulSet.yaml
是一个有状态应用，里面就定义了之前NFS的信息

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: jenkins
  labels:
    name: jenkins
  namespace: kube-ops
spec:
  serviceName: jenkins
  selector:
    matchLabels:
      app: jenkins
  replicas: 1
  updateStrategy:
    type: RollingUpdate
  template:
    metadata:
      name: jenkins
      labels:
        app: jenkins
    spec:
      terminationGracePeriodSeconds: 10
      serviceAccountName: jenkins
      containers:
        - name: jenkins
          image: jenkins/jenkins:lts-alpine
          imagePullPolicy: IfNotPresent
          ports:
          - containerPort: 8080
            name: web
            protocol: TCP
          - containerPort: 50000
            name: agent
            protocol: TCP
          resources:
            limits:
              cpu: 1
              memory: 1Gi
            requests:
              cpu: 0.5
              memory: 500Mi
          env:
            - name: LIMITS_MEMORY
              valueFrom:
                resourceFieldRef:
                  resource: limits.memory
                  divisor: 1Mi
            - name: JAVA_OPTS
              value: -Xmx$(LIMITS_MEMORY)m -XshowSettings:vm -Dhudson.slaves.NodeProvisioner.initialDelay=0 -Dhudson.slaves.NodeProvisioner.MARGIN=50 -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85
          volumeMounts:
            - name: jenkins-home
              mountPath: /var/jenkins_home
          livenessProbe:
            httpGet:
              path: /login
              port: 8080
            initialDelaySeconds: 60
            timeoutSeconds: 5
            failureThreshold: 12
          readinessProbe:
            httpGet:
              path: /login
              port: 8080
            initialDelaySeconds: 60
            timeoutSeconds: 5
            failureThreshold: 12
      securityContext:
        fsGroup: 1000
  volumeClaimTemplates:
  - metadata:
      name: jenkins-home
    spec:
      storageClassName: "managed-nfs-storage"
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 1Gi

Service.yaml
对外暴露一些端口

apiVersion: v1
kind: Service
metadata:
  name: jenkins
  namespace: kube-ops
  labels:
    app: jenkins
spec:
  selector:
    app: jenkins
  type: NodePort
  ports:
  - name: web
    port: 8080
    targetPort: web
  - name: agent
    port: 50000
    targetPort: agent

cd jenkins-master/
[root@master jenkins-master]# ls
rbac.yaml  ServiceaAcount.yaml  Service.yaml  StatefulSet.yaml

创建一个新的命名空间
kubectl create namespace kube-ops 

创建jenkins主节点的pod
kubectl create -f .


[root@master jenkins-master]# kubectl create namespace kube-ops
namespace/kube-ops created
[root@master jenkins-master]# kubectl create -f .
service/jenkins created
serviceaccount/jenkins created
statefulset.apps/jenkins created
role.rbac.authorization.k8s.io/jenkins created
rolebinding.rbac.authorization.k8s.io/jenkins created
clusterrole.rbac.authorization.k8s.io/jenkinsClusterRole created
rolebinding.rbac.authorization.k8s.io/jenkinsClusterRuleBinding created

我们可以通过很多个命令来查看pod和service的信息
查看的时候需要加上我们自己的命名空间
kubectl get pod --namespace kube-ops

[root@master jenkins-master]# kubectl get pod --namespace kube-ops
NAME        READY   STATUS    RESTARTS   AGE
jenkins-0   1/1     Running   0          7m38s

查看服务的信息
kubectl get service --namespace kube-ops

[root@master jenkins-master]# kubectl get service --namespace kube-ops
NAME      TYPE       CLUSTER-IP       EXTERNAL-IP   PORT(S)                          AGE
jenkins   NodePort   10.101.126.122   <none>        8080:30942/TCP,50000:31796/TCP   8m2s

-o wide查看更详细的信息
kubectl get service --namespace kube-ops -o wide

[root@master jenkins-master]# kubectl get service --namespace kube-ops -o wide
NAME      TYPE       CLUSTER-IP       EXTERNAL-IP   PORT(S)                          AGE     SELECTOR
jenkins   NodePort   10.101.126.122   <none>        8080:30942/TCP,50000:31796/TCP   8m20s   app=jenkins

查看pod更详细的信息
kubectl get pod --namespace kube-ops -o wide

[root@master jenkins-master]# kubectl get pod --namespace kube-ops -o wide
NAME        READY   STATUS    RESTARTS   AGE     IP               NODE    NOMINATED NODE   READINESS GATES
jenkins-0   1/1     Running   0          8m48s   10.244.166.129   node1   <none>           <none>

通过详细信息发现，我们的jenkins是部署在node1节点上
我们可以通过浏览器访问
node1的ip+端口

在这里插入图片描述
这里需要解锁jenkins，我们需要去jenkins的共享目录，NFS下寻找

nfs目录在/opt/nfs
[root@master jenkins-master]# cd /opt//nfs/
[root@master nfs]# ls
jenkins
[root@master nfs]# cd jenkins/
[root@master jenkins]# ll
total 4
drwxrwxrwx 13 root root 4096 May 13 16:26 kube-ops-jenkins-home-jenkins-0-pvc-0d59af09-0339-46eb-a6a0-550c40365efb

这个就是jenkins的运行目录
[root@master jenkins]# cd kube-ops-jenkins-home-jenkins-0-pvc-0d59af09-0339-46eb-a6a0-550c40365efb/
[root@master kube-ops-jenkins-home-jenkins-0-pvc-0d59af09-0339-46eb-a6a0-550c40365efb]# ls
config.xml                     jenkins.telemetry.Correlator.xml  nodes                     secrets      war
copy_reference_file.log        jobs                              plugins                   updates
hudson.model.UpdateCenter.xml  logs                              secret.key                userContent
identity.key.enc               nodeMonitors.xml                  secret.key.not-so-secret  users
[root@master kube-ops-jenkins-home-jenkins-0-pvc-0d59af09-0339-46eb-a6a0-550c40365efb]# cat secrets/initialAdminPassword 
5496a0ee3afd449fb65c709d6d721c5d

这个就是管理员密码 将其复制到浏览器解锁jenkins

跳过插件安装，我们一会儿修改下载源之后再安装插件
在这里插入图片描述
选择无

创建用户

Jenkins在k8s的环境下运行成功

Jenkins插件管理

Jenkins国外官方插件地址下载速度非常慢，所以可以修改为国内插件地址
Jenkins->Manage Jenkins->Manage Plugins，点击Available
在这里插入图片描述
这样做是为了把Jenkins官方的插件列表下载到本地，接着修改地址文件，替换为国内插件地址

[root@master kube-ops-jenkins-home-jenkins-0-pvc-0d59af09-0339-46eb-a6a0-550c40365efb]# pwd
/opt/nfs/jenkins/kube-ops-jenkins-home-jenkins-0-pvc-0d59af09-0339-46eb-a6a0-550c40365efb
[root@master kube-ops-jenkins-home-jenkins-0-pvc-0d59af09-0339-46eb-a6a0-550c40365efb]# cd updates/
[root@master updates]# ls
default.json  hudson.tasks.Maven.MavenInstaller

default.json就是插件下载地址
我们修改插件下载地址

sed -i 's/http:\/\/updates.jenkins-ci.org\/download/https:\/\/mirrors.tuna.tsinghua.edu.cn\/jenkins/g' default.json && sed -i 's/http:\/\/www.google.com/https:\/\/www.baidu.com/g' default.json

sed -i 's/http:\/\/updates.jenkins-ci.org\/download/https:\/\/mirrors.tuna.tsinghua.edu.cn\/jenkins/g'default.json && sed -i 's/http:\/\/www.google.com/https:\/\/www.baidu.com/g' default.json

最后，Manage Plugins点击Advanced，把Update Site改为国内插件下载地址

https://mirrors.tuna.tsinghua.edu.cn/jenkins/updates/update-center.json

在这里插入图片描述
在浏览器ip后面/restart 然后重启jenkins

安装基本插件

Chinese
Git
Pipeline
Extended Choice Parameter

等待安装
在这里插入图片描述

Jenkins与Kubernetes整合

安装Kubernetes插件
在这里插入图片描述

安装完成之后前往全局配置最下面
在这里插入图片描述

kubernetes地址采用了kube的服务器发现：https://kubernetes.default.svc.cluster.local
namespace填kube-ops，然后点击Test Connection，如果出现 Connection test successful 的提示信息证明 Jenkins 已经可以和 Kubernetes 系统正常通信
Jenkins URL 地址：http://jenkins.kube-ops.svc.cluster.local:8080

在这里插入图片描述
点击连接测试

在这里插入图片描述

构建Jenkins-Slave自定义镜像

Jenkins-Master在构建Job的时候，Kubernetes会创建Jenkins-Slave的Pod来完成Job的构建。我们选择运行Jenkins-Slave的镜像为官方推荐镜像：jenkins/jnlp-slave:latest，但是这个镜像里面并没有Maven环境，为了方便使用，我们需要自定义一个新的镜像

所以我们要编写Dockerfile自定义一个镜像

vim Dockerfile

FROM jenkins/jnlp-slave:latest

MAINTAINER xiaotian

# 切换到 root 账户进行操作
USER root

# 安装 maven
COPY apache-maven-3.6.2-bin.tar.gz .

RUN tar -zxf apache-maven-3.6.2-bin.tar.gz && \
    mv apache-maven-3.6.2 /usr/local && \
    rm -f apache-maven-3.6.2-bin.tar.gz && \
    ln -s /usr/local/apache-maven-3.6.2/bin/mvn /usr/bin/mvn && \
    ln -s /usr/local/apache-maven-3.6.2 /usr/local/apache-maven && \
    mkdir -p /usr/local/apache-maven/repo

COPY settings.xml /usr/local/apache-maven/conf/settings.xml

USER jenkins

还需要一个带有阿里源镜像的settings文件

<?xml version="1.0" encoding="UTF-8"?>

<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements.  See the NOTICE file
distributed with this work for additional information
regarding copyright ownership.  The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License.  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied.  See the License for the
specific language governing permissions and limitations
under the License.
-->

<!--
 | This is the configuration file for Maven. It can be specified at two levels:
 |
 |  1. User Level. This settings.xml file provides configuration for a single user,
 |                 and is normally provided in ${user.home}/.m2/settings.xml.
 |
 |                 NOTE: This location can be overridden with the CLI option:
 |
 |                 -s /path/to/user/settings.xml
 |
 |  2. Global Level. This settings.xml file provides configuration for all Maven
 |                 users on a machine (assuming they're all using the same Maven
 |                 installation). It's normally provided in
 |                 ${maven.conf}/settings.xml.
 |
 |                 NOTE: This location can be overridden with the CLI option:
 |
 |                 -gs /path/to/global/settings.xml
 |
 | The sections in this sample file are intended to give you a running start at
 | getting the most out of your Maven installation. Where appropriate, the default
 | values (values used when the setting is not specified) are provided.
 |
 |-->
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
  <!-- localRepository
   | The path to the local repository maven will use to store artifacts.
   |
   | Default: ${user.home}/.m2/repository
  <localRepository>/path/to/local/repo</localRepository>
  -->
  <localRepository>/usr/local/apache-maven/repo</localRepository>

  <!-- interactiveMode
   | This will determine whether maven prompts you when it needs input. If set to false,
   | maven will use a sensible default value, perhaps based on some other setting, for
   | the parameter in question.
   |
   | Default: true
  <interactiveMode>true</interactiveMode>
  -->

  <!-- offline
   | Determines whether maven should attempt to connect to the network when executing a build.
   | This will have an effect on artifact downloads, artifact deployment, and others.
   |
   | Default: false
  <offline>false</offline>
  -->

  <!-- pluginGroups
   | This is a list of additional group identifiers that will be searched when resolving plugins by their prefix, i.e.
   | when invoking a command line like "mvn prefix:goal". Maven will automatically add the group identifiers
   | "org.apache.maven.plugins" and "org.codehaus.mojo" if these are not already contained in the list.
   |-->
  <pluginGroups>
    <!-- pluginGroup
     | Specifies a further group identifier to use for plugin lookup.
    <pluginGroup>com.your.plugins</pluginGroup>
    -->
  </pluginGroups>

  <!-- proxies
   | This is a list of proxies which can be used on this machine to connect to the network.
   | Unless otherwise specified (by system property or command-line switch), the first proxy
   | specification in this list marked as active will be used.
   |-->
  <proxies>
    <!-- proxy
     | Specification for one proxy, to be used in connecting to the network.
     |
    <proxy>
      <id>optional</id>
      <active>true</active>
      <protocol>http</protocol>
      <username>proxyuser</username>
      <password>proxypass</password>
      <host>proxy.host.net</host>
      <port>80</port>
      <nonProxyHosts>local.net|some.host.com</nonProxyHosts>
    </proxy>
    -->
  </proxies>

  <!-- servers
   | This is a list of authentication profiles, keyed by the server-id used within the system.
   | Authentication profiles can be used whenever maven must make a connection to a remote server.
   |-->
  <servers>
    <!-- server
     | Specifies the authentication information to use when connecting to a particular server, identified by
     | a unique name within the system (referred to by the 'id' attribute below).
     |
     | NOTE: You should either specify username/password OR privateKey/passphrase, since these pairings are
     |       used together.
     |
    <server>
      <id>deploymentRepo</id>
      <username>repouser</username>
      <password>repopwd</password>
    </server>
    -->

    <!-- Another sample, using keys to authenticate.
    <server>
      <id>siteServer</id>
      <privateKey>/path/to/private/key</privateKey>
      <passphrase>optional; leave empty if not used.</passphrase>
    </server>
    -->
  </servers>

  <!-- mirrors
   | This is a list of mirrors to be used in downloading artifacts from remote repositories.
   |
   | It works like this: a POM may declare a repository to use in resolving certain artifacts.
   | However, this repository may have problems with heavy traffic at times, so people have mirrored
   | it to several places.
   |
   | That repository definition will have a unique id, so we can create a mirror reference for that
   | repository, to be used as an alternate download site. The mirror site will be the preferred
   | server for that repository.
   |-->
  <mirrors>
    <!-- mirror
     | Specifies a repository mirror site to use instead of a given repository. The repository that
     | this mirror serves has an ID that matches the mirrorOf element of this mirror. IDs are used
     | for inheritance and direct lookup purposes, and must be unique across the set of mirrors.
     |
    <mirror>
      <id>mirrorId</id>
      <mirrorOf>repositoryId</mirrorOf>
      <name>Human Readable Name for this Mirror.</name>
      <url>http://my.repository.com/repo/path</url>
    </mirror>
     -->
    <mirror>     
      <id>central</id>     
      <mirrorOf>central</mirrorOf>     
      <name>aliyun maven</name>
      <url>https://maven.aliyun.com/repository/public</url>     
    </mirror>
  </mirrors>

  <!-- profiles
   | This is a list of profiles which can be activated in a variety of ways, and which can modify
   | the build process. Profiles provided in the settings.xml are intended to provide local machine-
   | specific paths and repository locations which allow the build to work in the local environment.
   |
   | For example, if you have an integration testing plugin - like cactus - that needs to know where
   | your Tomcat instance is installed, you can provide a variable here such that the variable is
   | dereferenced during the build process to configure the cactus plugin.
   |
   | As noted above, profiles can be activated in a variety of ways. One way - the activeProfiles
   | section of this document (settings.xml) - will be discussed later. Another way essentially
   | relies on the detection of a system property, either matching a particular value for the property,
   | or merely testing its existence. Profiles can also be activated by JDK version prefix, where a
   | value of '1.4' might activate a profile when the build is executed on a JDK version of '1.4.2_07'.
   | Finally, the list of active profiles can be specified directly from the command line.
   |
   | NOTE: For profiles defined in the settings.xml, you are restricted to specifying only artifact
   |       repositories, plugin repositories, and free-form properties to be used as configuration
   |       variables for plugins in the POM.
   |
   |-->
  <profiles>
    <!-- profile
     | Specifies a set of introductions to the build process, to be activated using one or more of the
     | mechanisms described above. For inheritance purposes, and to activate profiles via <activatedProfiles/>
     | or the command line, profiles have to have an ID that is unique.
     |
     | An encouraged best practice for profile identification is to use a consistent naming convention
     | for profiles, such as 'env-dev', 'env-test', 'env-production', 'user-jdcasey', 'user-brett', etc.
     | This will make it more intuitive to understand what the set of introduced profiles is attempting
     | to accomplish, particularly when you only have a list of profile id's for debug.
     |
     | This profile example uses the JDK version to trigger activation, and provides a JDK-specific repo.
    <profile>
      <id>jdk-1.4</id>

      <activation>
        <jdk>1.4</jdk>
      </activation>

      <repositories>
        <repository>
          <id>jdk14</id>
          <name>Repository for JDK 1.4 builds</name>
          <url>http://www.myhost.com/maven/jdk14</url>
          <layout>default</layout>
          <snapshotPolicy>always</snapshotPolicy>
        </repository>
      </repositories>
    </profile>
    -->

    <!--
     | Here is another profile, activated by the system property 'target-env' with a value of 'dev',
     | which provides a specific path to the Tomcat instance. To use this, your plugin configuration
     | might hypothetically look like:
     |
     | ...
     | <plugin>
     |   <groupId>org.myco.myplugins</groupId>
     |   <artifactId>myplugin</artifactId>
     |
     |   <configuration>
     |     <tomcatLocation>${tomcatPath}</tomcatLocation>
     |   </configuration>
     | </plugin>
     | ...
     |
     | NOTE: If you just wanted to inject this configuration whenever someone set 'target-env' to
     |       anything, you could just leave off the <value/> inside the activation-property.
     |
    <profile>
      <id>env-dev</id>

      <activation>
        <property>
          <name>target-env</name>
          <value>dev</value>
        </property>
      </activation>

      <properties>
        <tomcatPath>/path/to/tomcat/instance</tomcatPath>
      </properties>
    </profile>
    -->
  </profiles>

  <!-- activeProfiles
   | List of profiles that are active for all builds.
   |
  <activeProfiles>
    <activeProfile>alwaysActiveProfile</activeProfile>
    <activeProfile>anotherAlwaysActiveProfile</activeProfile>
  </activeProfiles>
  -->
</settings>

最后需要一个maven的安装包
我这里使用 apache-maven-3.6.2-bin.tar

[root@master jenkins-slave]# ls
apache-maven-3.6.2-bin.tar.gz  Dockerfile  settings.xml

创建镜像
docker build -t jenkins-slave-maven:latest .

在这里插入图片描述

查看镜像
docker images

在这里插入图片描述
然后把镜像上传到Harbor的公共库library中（因为k8s从节点都需要下载此镜像，因此放在公共库最方便）

打标签
docker tag jenkins-slave-maven:latest 192.168.188.119:85/library/jenkins-slave-maven:latest

vim /etc/docker/daemon.json

 {
   "registry-mirrors": ["https://m0rsqupc.mirror.aliyuncs.com"],
   "insecure-registries": ["192.168.188.119:85"]
 }

登录
docker login -u admin -p Harbor12345 192.168.188.119:85

上传镜像
docker push 192.168.188.119:85/library/jenkins-slave-maven:latest

镜像上传成功
在这里插入图片描述

测试Jenkins-Slave是否可以创建

创建一个Jenkins流水线项目
在这里插入图片描述

添加gitlab的凭据，一会儿流水线脚本需要使用
在这里插入图片描述

编写Pipeline，从GItlab拉取代码

def git_address = "http://47.108.13.86:82/maomao_group/tensquare_back.git" 
def git_auth = "bb5a1b2e-8dfa-4557-a79b-66d0f1f05f5c" 

//创建一个Pod的模板，label为jenkins-slave 
podTemplate(label: 'jenkins-slave', cloud: 'kubernetes', containers: [ 
    containerTemplate( 
        name: 'jnlp', 
        image: "192.168.188.119:85/library/jenkins-slave-maven:latest"
    ) 
  ]
)
{ 
  //引用jenkins-slave的pod模块来构建Jenkins-Slave的pod
  node("jenkins-slave"){ 
      // 第一步 
      stage('拉取代码'){ 
          checkout([$class: 'GitSCM', branches: [[name: 'master']], 
          userRemoteConfigs: [[credentialsId: "${git_auth}", url: "${git_address}"]]])
      } 
  }
}

podTemplate 就是k8s的pod模板
label 是模板名字
cloud 是云名字在全局配置里的Cloud里自己填写的
containerTemplate 指定在pod里运行的容器

构建成功

在这里插入图片描述
中间遇到的错误

所有k8s节点必须授权harbor仓库地址

vi /etc/docker/daemon.json

{
"registry-mirrors": ["https://zydiol88.mirror.aliyuncs.com"],
"insecure-registries": ["harbor-ip:85"]
}

Jenkins+Kubernetes+Docker完成微服务持续集成

拉取代码，构建镜像

创建NFS共享目录

让所有Jenkins-Slave构建指向NFS的Maven的共享仓库目录

mkdir -p /opt/nfs/maven
vi /etc/exports

/opt/nfs/jenkins        *(rw,no_root_squash)
/opt/nfs/maven          *(rw,no_root_squash)

systemctl restart nfs

showmount -e 192.168.188.116 
Export list for 192.168.188.116:
/opt/nfs/maven   *
/opt/nfs/jenkins *

创建项目，编写构建Pipeline

在流水线里添加多选项参数
在这里插入图片描述

再添加字符参数

保存

接着去添加harbor的凭证
在这里插入图片描述
编写流水线脚本

def git_address = "http://192.168.188.120:82/maomao_group/tensquare_back.git"
def git_auth = "bb5a1b2e-8dfa-4557-a79b-66d0f1f05f5c" 
//构建版本的名称 
def tag = "latest" 
//Harbor私服地址 
def harbor_url = "192.168.188.119:85" 
//Harbor的项目名称 
def harbor_project_name = "tensquare" 
//Harbor的凭证 
def harbor_auth = "4fe90544-8b7b-4f81-b282-d20a2eb6e437" 

podTemplate(label: 'jenkins-slave', cloud: 'kubernetes', containers: [ 
	containerTemplate( 
		name: 'jnlp',
		image: "192.168.188.119:85/library/jenkins-slave-maven:latest"
	),
	containerTemplate(
		name: 'docker',
		image: "docker:stable",
		ttyEnabled: true,
		command: 'cat'
	),
  ],
  volumes: [
    hostPathVolume(mountPath: '/var/run/docker.sock', hostPath:'/var/run/docker.sock'),
	nfsVolume(mountPath: '/usr/local/apache-maven/repo', serverAddress: '192.168.188.116' , serverPath: '/opt/nfs/maven'),
	],
)
{ 
  node("jenkins-slave"){
	   // 第一步
	   stage('拉取代码'){
		  checkout([$class: 'GitSCM', branches: [[name: '${branch}']],
		  userRemoteConfigs: [[credentialsId: "${git_auth}", url: "${git_address}"]]])
	   }
	   // 第二步
	   stage('公共工程代码编译'){
			//编译并安装公共工程
		  sh "mvn -f tensquare_common clean install"
	   }
	   // 第三步
	   stage('构建镜像，部署项目'){
			 //把选择的项目信息转为数组
			 def selectedProjects = "${project_name}".split(',')
			 
			 for(int i=0;i<selectedProjects.size();i++){
				//取出每个项目的名称和端口 
				def currentProject = selectedProjects[i];
				//项目名称
				def currentProjectName = currentProject.split('@')[0]
				//项目启动端口
				def currentProjectPort = currentProject.split('@')[1]
				
				//定义镜像名称
				def imageName = "${currentProjectName}:${tag}"
				
				//编译，构建本地镜像
				sh "mvn -f ${currentProjectName} clean package dockerfile:build"
				
				container('docker') {
				
					//给镜像打标签
					sh "docker tag ${imageName} ${harbor_url}/${harbor_project_name}/${imageName}"
					
					//登录Harbor，并上传镜像
					withCredentials([usernamePassword(credentialsId:"${harbor_auth}", passwordVariable: 'password', usernameVariable: 'username')])
				{
						  //登录
						  sh "docker login -u ${username} -p ${password} ${harbor_url}"
						  
						  //上传镜像
						  sh "docker push ${harbor_url}/${harbor_project_name}/${imageName}"
					}
					//删除本地镜像
					sh "docker rmi -f ${imageName}"
					sh "docker rmi -f ${harbor_url}/${harbor_project_name}/${imageName}"
				}
			}
		}
	}
}

注意：在构建过程会发现无法创建仓库目录，是因为NFS共享目录权限不足，需更改权限

只用在k8s主节点

chmod -R 777 /opt/nfs/maven

如果出现这种错误
在这里插入图片描述

是Docker命令执行权限问题，所有k8s服务器都需要执行

chmod 777 /var/run/docker.sock

需要手动上传父工程依赖到NFS的Maven共享仓库目录中

记录

在这里插入图片描述
脚本写好之后，都能错这么多次，真的太难了

错了只能看日志一步一步进行排错

因为这一步实在是太容易错了，我记录一下构建过程，以后遇到问题可以查看

Started by user maomao
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] podTemplate
[Pipeline] {
[Pipeline] node
Created Pod: kubernetes kube-ops/jenkins-slave-qv8g5-p9qlr
Agent jenkins-slave-qv8g5-p9qlr is provisioned from template jenkins-slave-qv8g5
---
apiVersion: "v1"
kind: "Pod"
metadata:
  annotations:
    buildUrl: "http://jenkins.kube-ops.svc.cluster.local:8080/job/tensquare__back/15/"
    runUrl: "job/tensquare__back/15/"
  labels:
    jenkins: "slave"
    jenkins/label-digest: "5059d2cd0054f9fe75d61f97723d98ab1a42d71a"
    jenkins/label: "jenkins-slave"
  name: "jenkins-slave-qv8g5-p9qlr"
spec:
  containers:
  - env:
    - name: "JENKINS_SECRET"
      value: "********"
    - name: "JENKINS_AGENT_NAME"
      value: "jenkins-slave-qv8g5-p9qlr"
    - name: "JENKINS_NAME"
      value: "jenkins-slave-qv8g5-p9qlr"
    - name: "JENKINS_AGENT_WORKDIR"
      value: "/home/jenkins/agent"
    - name: "JENKINS_URL"
      value: "http://jenkins.kube-ops.svc.cluster.local:8080/"
    image: "192.168.188.119:85/library/jenkins-slave-maven:latest"
    imagePullPolicy: "IfNotPresent"
    name: "jnlp"
    resources:
      limits: {}
      requests: {}
    tty: false
    volumeMounts:
    - mountPath: "/usr/local/apache-maven/repo"
      name: "volume-1"
      readOnly: false
    - mountPath: "/var/run/docker.sock"
      name: "volume-0"
      readOnly: false
    - mountPath: "/home/jenkins/agent"
      name: "workspace-volume"
      readOnly: false
  - command:
    - "cat"
    image: "docker:stable"
    imagePullPolicy: "IfNotPresent"
    name: "docker"
    resources:
      limits: {}
      requests: {}
    tty: true
    volumeMounts:
    - mountPath: "/usr/local/apache-maven/repo"
      name: "volume-1"
      readOnly: false
    - mountPath: "/var/run/docker.sock"
      name: "volume-0"
      readOnly: false
    - mountPath: "/home/jenkins/agent"
      name: "workspace-volume"
      readOnly: false
  nodeSelector:
    kubernetes.io/os: "linux"
  restartPolicy: "Never"
  volumes:
  - hostPath:
      path: "/var/run/docker.sock"
    name: "volume-0"
  - name: "volume-1"
    nfs:
      path: "/opt/nfs/maven"
      readOnly: false
      server: "192.168.188.116"
  - emptyDir:
      medium: ""
    name: "workspace-volume"

Running on jenkins-slave-qv8g5-p9qlr in /home/jenkins/agent/workspace/tensquare__back
[Pipeline] {
[Pipeline] stage
[Pipeline] { (拉取代码)
[Pipeline] checkout
The recommended git tool is: NONE
using credential bb5a1b2e-8dfa-4557-a79b-66d0f1f05f5c
Cloning the remote Git repository
Cloning repository http://192.168.188.120:82/maomao_group/tensquare_back.git
 > git init /home/jenkins/agent/workspace/tensquare__back # timeout=10
Fetching upstream changes from http://192.168.188.120:82/maomao_group/tensquare_back.git
 > git --version # timeout=10
 > git --version # 'git version 2.20.1'
using GIT_ASKPASS to set credentials gitlab-http-auth
 > git fetch --tags --force --progress -- http://192.168.188.120:82/maomao_group/tensquare_back.git +refs/heads/*:refs/remotes/origin/* # timeout=10
Avoid second fetch
Checking out Revision dba0faf11591dc9aa572e43bb0b5134b3ebf195e (origin/master)
 > git config remote.origin.url http://192.168.188.120:82/maomao_group/tensquare_back.git # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git rev-parse origin/master^{commit} # timeout=10
 > git config core.sparsecheckout # timeout=10
 > git checkout -f dba0faf11591dc9aa572e43bb0b5134b3ebf195e # timeout=10
Commit message: "牛"
 > git rev-list --no-walk dba0faf11591dc9aa572e43bb0b5134b3ebf195e # timeout=10
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (公共工程代码编译)
[Pipeline] sh
+ mvn -f tensquare_common clean install
[INFO] Scanning for projects...
[INFO] 
[INFO] -------------------< com.tensquare:tensquare_common >-------------------
[INFO] Building tensquare_common 1.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-clean-plugin:3.0.0:clean (default-clean) @ tensquare_common ---
[INFO] Deleting /home/jenkins/agent/workspace/tensquare__back/tensquare_common/target
[INFO] 
[INFO] --- maven-resources-plugin:3.0.1:resources (default-resources) @ tensquare_common ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/jenkins/agent/workspace/tensquare__back/tensquare_common/src/main/resources
[INFO] skip non existing resourceDirectory /home/jenkins/agent/workspace/tensquare__back/tensquare_common/src/main/resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @ tensquare_common ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 5 source files to /home/jenkins/agent/workspace/tensquare__back/tensquare_common/target/classes
[INFO] 
[INFO] --- maven-resources-plugin:3.0.1:testResources (default-testResources) @ tensquare_common ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/jenkins/agent/workspace/tensquare__back/tensquare_common/src/test/resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:testCompile (default-testCompile) @ tensquare_common ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ tensquare_common ---
[INFO] No tests to run.
[INFO] 
[INFO] --- maven-jar-plugin:3.0.2:jar (default-jar) @ tensquare_common ---
[INFO] Building jar: /home/jenkins/agent/workspace/tensquare__back/tensquare_common/target/tensquare_common-1.0-SNAPSHOT.jar
[INFO] 
[INFO] --- maven-install-plugin:2.5.2:install (default-install) @ tensquare_common ---
[INFO] Installing /home/jenkins/agent/workspace/tensquare__back/tensquare_common/target/tensquare_common-1.0-SNAPSHOT.jar to /usr/local/apache-maven/repo/com/tensquare/tensquare_common/1.0-SNAPSHOT/tensquare_common-1.0-SNAPSHOT.jar
[INFO] Installing /home/jenkins/agent/workspace/tensquare__back/tensquare_common/pom.xml to /usr/local/apache-maven/repo/com/tensquare/tensquare_common/1.0-SNAPSHOT/tensquare_common-1.0-SNAPSHOT.pom
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  4.162 s
[INFO] Finished at: 2021-05-13T15:27:00Z
[INFO] ------------------------------------------------------------------------
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (构建镜像，部署项目)
[Pipeline] sh
+ mvn -f tensquare_eureka_server clean package dockerfile:build
[INFO] Scanning for projects...
[INFO] 
[INFO] ---------------< com.tensquare:tensquare_eureka_server >----------------
[INFO] Building tensquare_eureka_server 1.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-clean-plugin:3.0.0:clean (default-clean) @ tensquare_eureka_server ---
[INFO] Deleting /home/jenkins/agent/workspace/tensquare__back/tensquare_eureka_server/target
[INFO] 
[INFO] --- maven-resources-plugin:3.0.1:resources (default-resources) @ tensquare_eureka_server ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 0 resource
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @ tensquare_eureka_server ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 1 source file to /home/jenkins/agent/workspace/tensquare__back/tensquare_eureka_server/target/classes
[INFO] 
[INFO] --- maven-resources-plugin:3.0.1:testResources (default-testResources) @ tensquare_eureka_server ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/jenkins/agent/workspace/tensquare__back/tensquare_eureka_server/src/test/resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:testCompile (default-testCompile) @ tensquare_eureka_server ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ tensquare_eureka_server ---
[INFO] No tests to run.
[INFO] 
[INFO] --- maven-jar-plugin:3.0.2:jar (default-jar) @ tensquare_eureka_server ---
[INFO] Building jar: /home/jenkins/agent/workspace/tensquare__back/tensquare_eureka_server/target/tensquare_eureka_server-1.0-SNAPSHOT.jar
[INFO] 
[INFO] --- spring-boot-maven-plugin:2.0.1.RELEASE:repackage (default) @ tensquare_eureka_server ---
[INFO] 
[INFO] --- dockerfile-maven-plugin:1.3.6:build (default-cli) @ tensquare_eureka_server ---
[INFO] Building Docker context /home/jenkins/agent/workspace/tensquare__back/tensquare_eureka_server
[INFO] 
[INFO] Image will be built as tensquare_eureka_server:latest
[INFO] 
[INFO] Step 1/5 : FROM openjdk:8-jdk-alpine
[INFO] 
[INFO] Pulling from library/openjdk
[INFO] Digest: sha256:94792824df2df33402f201713f932b58cb9de94a0cd524164a0f2283343547b3
[INFO] Status: Image is up to date for openjdk:8-jdk-alpine
[INFO]  ---> a3562aa0b991
[INFO] Step 2/5 : ARG JAR_FILE
[INFO] 
[INFO]  ---> Using cache
[INFO]  ---> a2a3b3df4f15
[INFO] Step 3/5 : COPY ${JAR_FILE} app.jar
[INFO] 
[INFO]  ---> 656a595f07ab
[INFO] Step 4/5 : EXPOSE 10086
[INFO] 
[INFO]  ---> Running in 868cfdbdd284
[INFO] Removing intermediate container 868cfdbdd284
[INFO]  ---> 4fa7a8297ad1
[INFO] Step 5/5 : ENTRYPOINT ["java","-jar","/app.jar"]
[INFO] 
[INFO]  ---> Running in bee52b92749b
[INFO] Removing intermediate container bee52b92749b
[INFO]  ---> d0ec04357240
[INFO] Successfully built d0ec04357240
[INFO] Successfully tagged tensquare_eureka_server:latest
[INFO] 
[INFO] Detected build of image with id d0ec04357240
[INFO] Building jar: /home/jenkins/agent/workspace/tensquare__back/tensquare_eureka_server/target/tensquare_eureka_server-1.0-SNAPSHOT-docker-info.jar
[INFO] Successfully built tensquare_eureka_server:latest
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  27.874 s
[INFO] Finished at: 2021-05-13T15:27:29Z
[INFO] ------------------------------------------------------------------------
[Pipeline] container
[Pipeline] {
[Pipeline] sh
+ docker tag tensquare_eureka_server:latest 192.168.188.119:85/tensquare/tensquare_eureka_server:latest
[Pipeline] withCredentials
Masking supported pattern matches of $username or $password
[Pipeline] {
[Pipeline] sh
Warning: A secret was passed to "sh" using Groovy String interpolation, which is insecure.
		 Affected argument(s) used the following variable(s): [password, username]
		 See https://jenkins.io/redirect/groovy-string-interpolation for details.
+ docker login -u **** -p **** 192.168.188.119:85
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
[Pipeline] sh
+ docker push 192.168.188.119:85/tensquare/tensquare_eureka_server:latest
The push refers to repository [192.168.188.119:85/tensquare/tensquare_eureka_server]
c8a7e30e2667: Preparing
ceaf9e1ebef5: Preparing
9b9b7f3d56a0: Preparing
f1b5933fe4b5: Preparing
ceaf9e1ebef5: Layer already exists
9b9b7f3d56a0: Layer already exists
f1b5933fe4b5: Layer already exists
c8a7e30e2667: Pushed
latest: digest: sha256:320a88f1b5efa46bc74643c156652ca724211fff07578c8733ac7208f2ffa8a9 size: 1159
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] sh
+ docker rmi -f tensquare_eureka_server:latest
Untagged: tensquare_eureka_server:latest
[Pipeline] sh
+ docker rmi -f 192.168.188.119:85/tensquare/tensquare_eureka_server:latest
Untagged: 192.168.188.119:85/tensquare/tensquare_eureka_server:latest
Untagged: 192.168.188.119:85/tensquare/tensquare_eureka_server@sha256:320a88f1b5efa46bc74643c156652ca724211fff07578c8733ac7208f2ffa8a9
Deleted: sha256:d0ec043572402382bd3610e7fe9c63642c838edb43f7a16d822c374d231f9e9a
Deleted: sha256:4fa7a8297ad170d46a9b5774b6f6adacd6276e9acae9a4cbbb918286a7617c67
Deleted: sha256:656a595f07ab8eb40c24f787cdff50cbf0082e450251cc007a6bb055e67824e2
Deleted: sha256:caa525b07e2d84ac4c0ef38c69f193cd72d094c285d2971b5017359b9c9cb1d5
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // podTemplate
[Pipeline] End of Pipeline
Finished: SUCCESS

微服务部署到k8s

需要把构建好的微服务镜像交给k8s的环境来进行运行，因此需要一个Kubernetes Continuous Deploy插件
这是一个持续集成的插件

在这里插入图片描述
修改后的流水线脚本

deploy.yml描述了在k8s里怎么部署不同微服务
kubeconfigId 是证书，需要在jenkins环境里创建出来

添加k8s证书，添加凭据
在这里插入图片描述
这个证书在k8s主节点

cd /root/.kube
cat config

显示的内容就是证书，我们需要把内容原封不动的全部复制过来

在这里插入图片描述
然后再次查看会产生一个id，这个id添加到脚本里

def git_address = "http://192.168.188.120:82/maomao_group/tensquare_back.git"
def git_auth = "bb5a1b2e-8dfa-4557-a79b-66d0f1f05f5c" 
//构建版本的名称 
def tag = "latest" 
//Harbor私服地址 
def harbor_url = "192.168.188.119:85" 
//Harbor的项目名称 
def harbor_project_name = "tensquare" 
//Harbor的凭证 
def harbor_auth = "2ec3c8b6-f9b6-4ef6-b4bc-fdba74f99420" 
//k8s的凭证
def k8s_auth = ""9565b450-3899-4892-8aed-d15b6f26f8fd

podTemplate(label: 'jenkins-slave', cloud: 'kubernetes', containers: [ 
	containerTemplate( 
		name: 'jnlp',
		image: "192.168.188.119:85/library/jenkins-slave-maven:latest"
	),
	containerTemplate(
		name: 'docker',
		image: "docker:stable",
		ttyEnabled: true,
		command: 'cat'
	),
  ],
  volumes: [
    hostPathVolume(mountPath: '/var/run/docker.sock', hostPath:'/var/run/docker.sock'),
	nfsVolume(mountPath: '/usr/local/apache-maven/repo', serverAddress: '192.168.188.116' , serverPath: '/opt/nfs/maven'),
	],
)
{ 
  node("jenkins-slave"){
	   // 第一步
	   stage('拉取代码'){
		  checkout([$class: 'GitSCM', branches: [[name: '${branch}']],
		  userRemoteConfigs: [[credentialsId: "${git_auth}", url: "${git_address}"]]])
	   }
	   // 第二步
	   stage('公共工程代码编译'){
			//编译并安装公共工程
		  sh "mvn -f tensquare_common clean install"
	   }
	   // 第三步
	   stage('构建镜像，部署项目'){
			 //把选择的项目信息转为数组
			 def selectedProjects = "${project_name}".split(',')
			 
			 for(int i=0;i<selectedProjects.size();i++){
				//取出每个项目的名称和端口 
				def currentProject = selectedProjects[i];
				//项目名称
				def currentProjectName = currentProject.split('@')[0]
				//项目启动端口
				def currentProjectPort = currentProject.split('@')[1]
				
				//定义镜像名称
				def imageName = "${currentProjectName}:${tag}"
				
				//编译，构建本地镜像
				sh "mvn -f ${currentProjectName} clean package dockerfile:build"
				
				container('docker') {
				
					//给镜像打标签
					sh "docker tag ${imageName} ${harbor_url}/${harbor_project_name}/${imageName}"
					
					//登录Harbor，并上传镜像
					withCredentials([usernamePassword(credentialsId:"${harbor_auth}", passwordVariable: 'password', usernameVariable: 'username')])
				{
						  //登录
						  sh "docker login -u ${username} -p ${password} ${harbor_url}"
						  
						  //上传镜像
						  sh "docker push ${harbor_url}/${harbor_project_name}/${imageName}"
					}
					//删除本地镜像
					sh "docker rmi -f ${imageName}"
					sh "docker rmi -f ${harbor_url}/${harbor_project_name}/${imageName}"
				}
				def deploy_image_name = "${harbor_url}/${harbor_project_name}/${imageName}"
				
				//部署到K8S
				sh """
						sed -i 's#\$IMAGE_NAME#${deploy_image_name}#' ${currentProjectName}/deploy.yml 
						sed -i 's#\$SECRET_NAME#${secret_name}#' ${currentProjectName}/deploy.yml 
				   """

				   kubernetesDeploy configs: "${currentProjectName}/deploy.yml", kubeconfigId: "${k8s_auth}"
				
			}
		}
	}
}

完成eureka集群部署到Kubernetes

我们在eureka项目下创建一个deploy.yml文件

---
apiVersion: v1
kind: Service
metadata:
  name: eureka
  labels:
    app: eureka
spec:
  type: NodePort
  ports:
    - port: 10086
      name: eureka
      targetPort: 10086
  selector:
    app: eureka
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: eureka
spec:
  serviceName: "eureka"
  replicas: 2
  selector:
    matchLabels:
      app: eureka
  template:
    metadata:
      labels:
        app: eureka
    spec:
      imagePullSecrets:
        - name: $SECRET_NAME
      containers:
        - name: eureka
          image: $IMAGE_NAME
          ports:
            - containerPort: 10086
          env:
            - name: MY_POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: EUREKA_SERVER
              value: "http://eureka-0.eureka:10086/eureka/,http://eureka-1.eureka:10086/eureka/"
            - name: EUREKA_INSTANCE_HOSTNAME
              value: ${MY_POD_NAME}.eureka
  podManagementPolicy: "Parallel"

接着更改eruka的配置文件application.yml

server:
  port: ${PORT:10086}
spring:
  application:
    name: eureka

eureka:
  server:
    # 续期时间，即扫描失效服务的间隔时间（缺省为60*1000ms）
    eviction-interval-timer-in-ms: 5000
    enable-self-preservation: false
    use-read-only-response-cache: false
  client:
    # eureka client间隔多久去拉取服务注册信息 默认30s
    registry-fetch-interval-seconds: 5
    serviceUrl:
      defaultZone: ${EUREKA_SERVER:http://127.0.0.1:${server.port}/eureka/}
  instance:
    # 心跳间隔时间，即发送一次心跳之后，多久在发起下一次（缺省为30s）
    lease-renewal-interval-in-seconds: 5
    # 在收到一次心跳之后，等待下一次心跳的空档时间，大于心跳间隔即可，即服务续约到期时间（缺省为90s）
    lease-expiration-duration-in-seconds: 10
    instance-id: ${EUREKA_INSTANCE_HOSTNAME:${spring.application.name}}:${server.port}@${random.long(1000000,9999999)}
    hostname: ${EUREKA_INSTANCE_HOSTNAME:${spring.application.name}}

提交代码到仓库
在这里插入图片描述
尝试构建eruka服务器，但报错显示找不到secret_name这个属性

修改脚本定义一个secret_name的变量

然后我们去制作凭证

首先在k8s主节点登录harbor仓库
docker login -u maomao -p Xiaotian123 192.168.188.119:85

然后使用一串命令生成证书
kubectl create secret docker-registry registry-auth-secret --docker-server=192.168.188.119:85 --docker-username=maomao --docker-password=Xiaotian123 --docker-email=123@qq.com

在这里插入图片描述
查看证书

kubectl get secret

在这里插入图片描述
然后再次构建eruka服务器

构建成功之后，可以在主节点查看信息

在这里插入图片描述

kubectl get pods

查看端口
kubectl get svc

在这里插入图片描述

我们可以通过端口访问所有k8s服务器ip

在这里插入图片描述

错误排查

在这里插入图片描述
经过我的深思熟虑，发现是密钥的问题
因为直接复制，导致有些地方本来是一行的结果变成了两行

将换行删除就行了

结果又错了
在这里插入图片描述
我干脆直接把密钥文件拷贝到自己电脑上进行复制，这样格式就不会错了

记录一下构建过程

Started by user maomao
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] podTemplate
[Pipeline] {
[Pipeline] node
Created Pod: kubernetes kube-ops/jenkins-slave-dlxsx-7pxm1
Agent jenkins-slave-dlxsx-7pxm1 is provisioned from template jenkins-slave-dlxsx
---
apiVersion: "v1"
kind: "Pod"
metadata:
  annotations:
    buildUrl: "http://jenkins.kube-ops.svc.cluster.local:8080/job/tensquare__back/20/"
    runUrl: "job/tensquare__back/20/"
  labels:
    jenkins: "slave"
    jenkins/label-digest: "5059d2cd0054f9fe75d61f97723d98ab1a42d71a"
    jenkins/label: "jenkins-slave"
  name: "jenkins-slave-dlxsx-7pxm1"
spec:
  containers:
  - env:
    - name: "JENKINS_SECRET"
      value: "********"
    - name: "JENKINS_AGENT_NAME"
      value: "jenkins-slave-dlxsx-7pxm1"
    - name: "JENKINS_NAME"
      value: "jenkins-slave-dlxsx-7pxm1"
    - name: "JENKINS_AGENT_WORKDIR"
      value: "/home/jenkins/agent"
    - name: "JENKINS_URL"
      value: "http://jenkins.kube-ops.svc.cluster.local:8080/"
    image: "192.168.188.119:85/library/jenkins-slave-maven:latest"
    imagePullPolicy: "IfNotPresent"
    name: "jnlp"
    resources:
      limits: {}
      requests: {}
    tty: false
    volumeMounts:
    - mountPath: "/usr/local/apache-maven/repo"
      name: "volume-1"
      readOnly: false
    - mountPath: "/var/run/docker.sock"
      name: "volume-0"
      readOnly: false
    - mountPath: "/home/jenkins/agent"
      name: "workspace-volume"
      readOnly: false
  - command:
    - "cat"
    image: "docker:stable"
    imagePullPolicy: "IfNotPresent"
    name: "docker"
    resources:
      limits: {}
      requests: {}
    tty: true
    volumeMounts:
    - mountPath: "/usr/local/apache-maven/repo"
      name: "volume-1"
      readOnly: false
    - mountPath: "/var/run/docker.sock"
      name: "volume-0"
      readOnly: false
    - mountPath: "/home/jenkins/agent"
      name: "workspace-volume"
      readOnly: false
  nodeSelector:
    kubernetes.io/os: "linux"
  restartPolicy: "Never"
  volumes:
  - hostPath:
      path: "/var/run/docker.sock"
    name: "volume-0"
  - name: "volume-1"
    nfs:
      path: "/opt/nfs/maven"
      readOnly: false
      server: "192.168.188.116"
  - emptyDir:
      medium: ""
    name: "workspace-volume"

Running on jenkins-slave-dlxsx-7pxm1 in /home/jenkins/agent/workspace/tensquare__back
[Pipeline] {
[Pipeline] stage
[Pipeline] { (拉取代码)
[Pipeline] checkout
The recommended git tool is: NONE
using credential bb5a1b2e-8dfa-4557-a79b-66d0f1f05f5c
Cloning the remote Git repository
Cloning repository http://192.168.188.120:82/maomao_group/tensquare_back.git
 > git init /home/jenkins/agent/workspace/tensquare__back # timeout=10
Fetching upstream changes from http://192.168.188.120:82/maomao_group/tensquare_back.git
 > git --version # timeout=10
 > git --version # 'git version 2.20.1'
using GIT_ASKPASS to set credentials gitlab-http-auth
 > git fetch --tags --force --progress -- http://192.168.188.120:82/maomao_group/tensquare_back.git +refs/heads/*:refs/remotes/origin/* # timeout=10
Avoid second fetch
Checking out Revision a35a2f630f44b425c52aa483aef1b7dc64539940 (origin/master)
 > git config remote.origin.url http://192.168.188.120:82/maomao_group/tensquare_back.git # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git rev-parse origin/master^{commit} # timeout=10
 > git config core.sparsecheckout # timeout=10
 > git checkout -f a35a2f630f44b425c52aa483aef1b7dc64539940 # timeout=10
Commit message: "K8S"
 > git rev-list --no-walk a35a2f630f44b425c52aa483aef1b7dc64539940 # timeout=10
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (公共工程代码编译)
[Pipeline] sh
+ mvn -f tensquare_common clean install
[INFO] Scanning for projects...
[INFO] 
[INFO] -------------------< com.tensquare:tensquare_common >-------------------
[INFO] Building tensquare_common 1.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-clean-plugin:3.0.0:clean (default-clean) @ tensquare_common ---
[INFO] Deleting /home/jenkins/agent/workspace/tensquare__back/tensquare_common/target
[INFO] 
[INFO] --- maven-resources-plugin:3.0.1:resources (default-resources) @ tensquare_common ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/jenkins/agent/workspace/tensquare__back/tensquare_common/src/main/resources
[INFO] skip non existing resourceDirectory /home/jenkins/agent/workspace/tensquare__back/tensquare_common/src/main/resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @ tensquare_common ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 5 source files to /home/jenkins/agent/workspace/tensquare__back/tensquare_common/target/classes
[INFO] 
[INFO] --- maven-resources-plugin:3.0.1:testResources (default-testResources) @ tensquare_common ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/jenkins/agent/workspace/tensquare__back/tensquare_common/src/test/resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:testCompile (default-testCompile) @ tensquare_common ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ tensquare_common ---
[INFO] No tests to run.
[INFO] 
[INFO] --- maven-jar-plugin:3.0.2:jar (default-jar) @ tensquare_common ---
[INFO] Building jar: /home/jenkins/agent/workspace/tensquare__back/tensquare_common/target/tensquare_common-1.0-SNAPSHOT.jar
[INFO] 
[INFO] --- maven-install-plugin:2.5.2:install (default-install) @ tensquare_common ---
[INFO] Installing /home/jenkins/agent/workspace/tensquare__back/tensquare_common/target/tensquare_common-1.0-SNAPSHOT.jar to /usr/local/apache-maven/repo/com/tensquare/tensquare_common/1.0-SNAPSHOT/tensquare_common-1.0-SNAPSHOT.jar
[INFO] Installing /home/jenkins/agent/workspace/tensquare__back/tensquare_common/pom.xml to /usr/local/apache-maven/repo/com/tensquare/tensquare_common/1.0-SNAPSHOT/tensquare_common-1.0-SNAPSHOT.pom
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  4.069 s
[INFO] Finished at: 2021-05-13T16:34:40Z
[INFO] ------------------------------------------------------------------------
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (构建镜像，部署项目)
[Pipeline] sh
+ mvn -f tensquare_eureka_server clean package dockerfile:build
[INFO] Scanning for projects...
[INFO] 
[INFO] ---------------< com.tensquare:tensquare_eureka_server >----------------
[INFO] Building tensquare_eureka_server 1.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-clean-plugin:3.0.0:clean (default-clean) @ tensquare_eureka_server ---
[INFO] Deleting /home/jenkins/agent/workspace/tensquare__back/tensquare_eureka_server/target
[INFO] 
[INFO] --- maven-resources-plugin:3.0.1:resources (default-resources) @ tensquare_eureka_server ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 0 resource
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @ tensquare_eureka_server ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 1 source file to /home/jenkins/agent/workspace/tensquare__back/tensquare_eureka_server/target/classes
[INFO] 
[INFO] --- maven-resources-plugin:3.0.1:testResources (default-testResources) @ tensquare_eureka_server ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/jenkins/agent/workspace/tensquare__back/tensquare_eureka_server/src/test/resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:testCompile (default-testCompile) @ tensquare_eureka_server ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ tensquare_eureka_server ---
[INFO] No tests to run.
[INFO] 
[INFO] --- maven-jar-plugin:3.0.2:jar (default-jar) @ tensquare_eureka_server ---
[INFO] Building jar: /home/jenkins/agent/workspace/tensquare__back/tensquare_eureka_server/target/tensquare_eureka_server-1.0-SNAPSHOT.jar
[INFO] 
[INFO] --- spring-boot-maven-plugin:2.0.1.RELEASE:repackage (default) @ tensquare_eureka_server ---
[INFO] 
[INFO] --- dockerfile-maven-plugin:1.3.6:build (default-cli) @ tensquare_eureka_server ---
[INFO] Building Docker context /home/jenkins/agent/workspace/tensquare__back/tensquare_eureka_server
[INFO] 
[INFO] Image will be built as tensquare_eureka_server:latest
[INFO] 
[INFO] Step 1/5 : FROM openjdk:8-jdk-alpine
[INFO] 
[INFO] Pulling from library/openjdk
[INFO] Digest: sha256:94792824df2df33402f201713f932b58cb9de94a0cd524164a0f2283343547b3
[INFO] Status: Image is up to date for openjdk:8-jdk-alpine
[INFO]  ---> a3562aa0b991
[INFO] Step 2/5 : ARG JAR_FILE
[INFO] 
[INFO]  ---> Using cache
[INFO]  ---> a2a3b3df4f15
[INFO] Step 3/5 : COPY ${JAR_FILE} app.jar
[INFO] 
[INFO]  ---> 1b476689026f
[INFO] Step 4/5 : EXPOSE 10086
[INFO] 
[INFO]  ---> Running in 68e029b5bdac
[INFO] Removing intermediate container 68e029b5bdac
[INFO]  ---> f76e827eb058
[INFO] Step 5/5 : ENTRYPOINT ["java","-jar","/app.jar"]
[INFO] 
[INFO]  ---> Running in 92ad4dcb6596
[INFO] Removing intermediate container 92ad4dcb6596
[INFO]  ---> ce5f4598f452
[INFO] Successfully built ce5f4598f452
[INFO] Successfully tagged tensquare_eureka_server:latest
[INFO] 
[INFO] Detected build of image with id ce5f4598f452
[INFO] Building jar: /home/jenkins/agent/workspace/tensquare__back/tensquare_eureka_server/target/tensquare_eureka_server-1.0-SNAPSHOT-docker-info.jar
[INFO] Successfully built tensquare_eureka_server:latest
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  28.550 s
[INFO] Finished at: 2021-05-13T16:35:10Z
[INFO] ------------------------------------------------------------------------
[Pipeline] container
[Pipeline] {
[Pipeline] sh
+ docker tag tensquare_eureka_server:latest 192.168.188.119:85/tensquare/tensquare_eureka_server:latest
[Pipeline] withCredentials
Masking supported pattern matches of $username or $password
[Pipeline] {
[Pipeline] sh
Warning: A secret was passed to "sh" using Groovy String interpolation, which is insecure.
		 Affected argument(s) used the following variable(s): [password, username]
		 See https://jenkins.io/redirect/groovy-string-interpolation for details.
+ docker login -u **** -p **** 192.168.188.119:85
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
[Pipeline] sh
+ docker push 192.168.188.119:85/tensquare/tensquare_eureka_server:latest
The push refers to repository [192.168.188.119:85/tensquare/tensquare_eureka_server]
6b8a8e1926e4: Preparing
ceaf9e1ebef5: Preparing
9b9b7f3d56a0: Preparing
f1b5933fe4b5: Preparing
f1b5933fe4b5: Layer already exists
9b9b7f3d56a0: Layer already exists
ceaf9e1ebef5: Layer already exists
6b8a8e1926e4: Pushed
latest: digest: sha256:584ab9d6bd684636dfef55c2a2ac3d36d445b287f5a0a89a6694823655f909b1 size: 1159
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] sh
+ docker rmi -f tensquare_eureka_server:latest
Untagged: tensquare_eureka_server:latest
[Pipeline] sh
+ docker rmi -f 192.168.188.119:85/tensquare/tensquare_eureka_server:latest
Untagged: 192.168.188.119:85/tensquare/tensquare_eureka_server:latest
Untagged: 192.168.188.119:85/tensquare/tensquare_eureka_server@sha256:584ab9d6bd684636dfef55c2a2ac3d36d445b287f5a0a89a6694823655f909b1
Deleted: sha256:ce5f4598f452e300f537deacab64ee958f93a7c39ced0ff71f360f9c4d5d7572
Deleted: sha256:f76e827eb0580d99d60f8709608850589404666611fae1068e4e2231d100e6a3
Deleted: sha256:1b476689026f842092261a4d524043b6c4aa754e6b7b68c25501269fa1e17dce
Deleted: sha256:731b8259434bd270843c034ed36011581a2371daf28f682a6e3d0fa8b713545a
[Pipeline] }
[Pipeline] // container
[Pipeline] sh
+ sed -i s#$IMAGE_NAME#192.168.188.119:85/tensquare/tensquare_eureka_server:latest# tensquare_eureka_server/deploy.yml
+ sed -i s#$SECRET_NAME#registry-auth-secret# tensquare_eureka_server/deploy.yml
[Pipeline] kubernetesDeploy
Starting Kubernetes deployment
Loading configuration: /home/jenkins/agent/workspace/tensquare__back/tensquare_eureka_server/deploy.yml
Finished Kubernetes deployment
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // podTemplate
[Pipeline] End of Pipeline
Finished: SUCCESS

网关集群部署到k8s

更改zull的配置，将地址换成这个

defaultZone: http://eureka-0.eureka:10086/eureka/,http://eureka-1.eureka:10086/eureka/

编写一个deploy.yml文件

---
apiVersion: v1
kind: Service
metadata:
  name: zuul
  labels:
    app: zuul
spec:
  type: NodePort
  ports:
    - port: 10020
      name: zuul
      targetPort: 10020
  selector:
    app: zuul
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: zuul
spec:
  serviceName: "zuul"
  replicas: 2
  selector:
    matchLabels:
      app: zuul
  template:
    metadata:
      labels:
        app: zuul
    spec:
      imagePullSecrets:
        - name: $SECRET_NAME
      containers:
        - name: zuul
          image: $IMAGE_NAME
          ports:
            - containerPort: 10020
  podManagementPolicy: "Parallel"

去进行任务构建
在这里插入图片描述
报错了

原来时需要手动上传父工程依赖到NFS的Maven共享仓库目录中

[root@master tensquare]# pwd
/opt/nfs/maven/com/tensquare

[root@master tensquare]# mv /root/tensquare_parent/ ./
[root@master tensquare]# ls
tensquare_common  tensquare_parent

上传完父工程依赖之后再次进行构建
在这里插入图片描述

已经有两个网关了
在这里插入图片描述

在这里插入图片描述

stan Z

关注

15
点赞
踩
53

收藏

觉得还不错? 一键收藏
32
评论
Kubernetes和Jenkins——基于Kubernetes构建Jenkins持续集成平台

基于Kubernetes/K8S构建Jenkins持续集成平台Jenkins的Master-Slave分布式构建Jenkins的Master-Slave分布式构建什么是Master-Slave分布式构建Jenkins的Master-Slave分布式构建，就是通过将构建过程分配到从属Slave节点上，从而减轻Master节点的压力，而且可以同时构建多个，类似负载均衡的概念。...
复制链接

扫一扫