[kubernetes]-kubernetes部署jenkins编译发布pipeline项目

1 创建jenkins-master镜像

资源下载地址：

链接: https://pan.baidu.com/s/1WUk8CTVkKYPGNz_ufDKtPw 密码: 1n94

如果资源有问题 可以直接下官方镜像 然后从里面scp出来

需要如下文件 jdk安装包 tomcat 里面包含需要的jenkins的war包 名称为

1 ROOT.war

2 Dockerfile

3 settings.xml(如果不需要在master上编译的话 不要也没关系)

4 .ssh 文件 用于下载ssh 的git代码 以及config文件 用于避免下载代码的时候交互yea/no 否则可能会卡在git clone

这里tomcat里的war包 记得重命名 否则如果是jenkins.war打好镜像之后 使用slave编译的时候会提示找不到标签为xxx的slave agent，看了pod报错是找不到api 之类的。url 直接是/的 不是/jenkins，不太好改。

Dockerfile

FROM centos:centos7.4.1708
MAINTAINER PDABC Enterprise Container Images <jiaminxu@hz-health.cn>

ADD jdk-8u161-linux-x64.tar.gz /usr/local/java/
COPY tomcat /usr/local/tomcat
COPY .ssh /root/.ssh
RUN cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime \
  && yum -y install git subversion sshpassi maven \
  && rm -rf /var/cache/yum

COPY settings.xml /etc/maven/settings.xml
WORKDIR /root
ENV JAVA_HOME /usr/local/java/jdk1.8.0_161
ENV JRE_HOME ${JAVA_HOME}/jre
ENV CLASSPATH .:${JAVA_HOME}/lib:${JRE_HOME}/lib
ENV PATH ${JAVA_HOME}/bin:$PATH


#公开端口
EXPOSE 8080 5000

USER root
#设置启动命令
ENTRYPOINT ["/usr/local/tomcat/bin/catalina.sh","run"]

docker build -t registry.cn-shanghai.aliyuncs.com/yjk-datag/jenkins:master01 .
docker push registry.cn-shanghai.aliyuncs.com/yjk-datag/jenkins:master01

为什么创建自己的master镜像 原因如下

[root@op-k8s-m001 jenkins]# kubectl logs -f  jenkins-6db584c4cd-lfl9g -n devops
VM settings:
    Max. Heap Size (Estimated): 247.50M
    Ergonomics Machine Class: server
    Using VM: OpenJDK 64-Bit Server VM

Running from: /usr/share/jenkins/jenkins.war
webroot: EnvVars.masterEnvVars.get("JENKINS_HOME")
2020-08-31 06:37:21.552+0000 [id=1]	INFO	org.eclipse.jetty.util.log.Log#initialized: Logging initialized @646ms to org.eclipse.jetty.util.log.JavaUtilLog
2020-08-31 06:37:21.832+0000 [id=1]	INFO	winstone.Logger#logInternal: Beginning extraction from war file
2020-08-31 06:37:23.518+0000 [id=1]	WARNING	o.e.j.s.handler.ContextHandler#setContextPath: Empty contextPath
2020-08-31 06:37:23.571+0000 [id=1]	INFO	org.eclipse.jetty.server.Server#doStart: jetty-9.4.z-SNAPSHOT; built: 2019-05-02T00:04:53.875Z; git: e1bc35120a6617ee3df052294e433f3a25ce7097; jvm 1.8.0_222-b10
2020-08-31 06:37:24.132+0000 [id=1]	INFO	o.e.j.w.StandardDescriptorProcessor#visitServlet: NO JSP Support for /, did not find org.eclipse.jetty.jsp.JettyJspServlet
2020-08-31 06:37:24.222+0000 [id=1]	INFO	o.e.j.s.s.DefaultSessionIdManager#doStart: DefaultSessionIdManager workerName=node0
2020-08-31 06:37:24.223+0000 [id=1]	INFO	o.e.j.s.s.DefaultSessionIdManager#doStart: No SessionScavenger set, using defaults
2020-08-31 06:37:24.225+0000 [id=1]	INFO	o.e.j.server.session.HouseKeeper#startScavenging: node0 Scavenging every 600000ms
Jenkins home directory: /var/jenkins_home found at: EnvVars.masterEnvVars.get("JENKINS_HOME")
2020-08-31 06:37:25.231+0000 [id=1]	INFO	o.e.j.s.handler.ContextHandler#doStart: Started w.@26be6ca7{Jenkins v2.199,/,file:///var/jenkins_home/war/,AVAILABLE}{/var/jenkins_home/war}
2020-08-31 06:37:25.247+0000 [id=1]	INFO	o.e.j.server.AbstractConnector#doStart: Started ServerConnector@19932c16{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}
2020-08-31 06:37:25.247+0000 [id=1]	INFO	org.eclipse.jetty.server.Server#doStart: Started @4342ms
2020-08-31 06:37:25.248+0000 [id=20]	INFO	winstone.Logger#logInternal: Winstone Servlet Engine v4.0 running: controlPort=disabled
2020-08-31 06:37:27.660+0000 [id=25]	INFO	jenkins.InitReactorRunner$1#onAttained: Started initialization
2020-08-31 06:37:29.064+0000 [id=26]	WARNING	hudson.ClassicPluginStrategy#createPluginWrapper: encountered /var/jenkins_home/plugins/changelog-environment.hpi under a nonstandard name; expected changelog-environment.jpi
2020-08-31 06:37:29.071+0000 [id=26]	INFO	jenkins.InitReactorRunner$1#onAttained: Listed all plugins
2020-08-31 06:37:30.038+0000 [id=26]	INFO	com.sic.plugins.kpp.KPPPlugin#start: starting kpp plugin
2020-08-31 06:37:30.065+0000 [id=26]	INFO	o.j.h.p.m.M2ReleaseBuildWrapper$DescriptorImpl#<clinit>: Using new style Permission with PermissionScope
2020-08-31 06:37:30.128+0000 [id=26]	SEVERE	jenkins.InitReactorRunner$1#onTaskFailed: Failed Loading plugin Pipeline Maven Integration Plugin v3.8.2 (pipeline-maven)
java.io.IOException: Pipeline Maven Integration Plugin version 3.8.2 failed to load.
 - HTML Publisher plugin version 1.14 is older than required. To fix, install version 1.16 or later.
 - Jenkins JaCoCo plugin version 2.2.1 is older than required. To fix, install version 3.0.3 or later.
	at hudson.PluginWrapper.resolvePluginDependencies(PluginWrapper.java:922)
	at hudson.PluginManager$2$1$1.run(PluginManager.java:546)
	at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169)
	at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296)
	at jenkins.model.Jenkins$5.runTask(Jenkins.java:1124)
	at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214)
	at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
2020-08-31 06:37:30.142+0000 [id=25]	INFO	ruby.RubyRuntimePlugin#start: Injecting JRuby into XStream
2020-08-31 06:37:55.440+0000 [id=26]	INFO	jenkins.InitReactorRunner$1#onAttained: Prepared all plugins
2020-08-31 06:37:55.730+0000 [id=26]	INFO	jenkins.InitReactorRunner$1#onAttained: Started all plugins
2020-08-31 06:37:57.349+0000 [id=26]	WARNING	h.ExtensionFinder$GuiceFinder$FaultTolerantScope$1#error: Failed to instantiate Key[type=jenkins.telemetry.Correlator, annotation=[none]]; skipping this component
com.google.inject.ProvisionException: Unable to provision, see the following errors:

1) Tried proxying jenkins.telemetry.Correlator to support a circular dependency, but it is not an interface.

1 error
	at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:52)
	at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:145)
	at hudson.ExtensionFinder$GuiceFinder$FaultTolerantScope$1.get(ExtensionFinder.java:439)
	at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41)
	at com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1016)
	at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1103)
	at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1012)
	at hudson.ExtensionFinder$GuiceFinder._find(ExtensionFinder.java:401)
	at hudson.ExtensionFinder$GuiceFinder.find(ExtensionFinder.java:392)
	at hudson.ClassicPluginStrategy.findComponents(ClassicPluginStrategy.java:344)
	at hudson.ExtensionList.load(ExtensionList.java:381)
	at hudson.ExtensionList.ensureLoaded(ExtensionList.java:317)
	at hudson.ExtensionList.getComponents(ExtensionList.java:183)
	at hudson.DescriptorExtensionList.load(DescriptorExtensionList.java:193)
	at hudson.ExtensionList.ensureLoaded(ExtensionList.java:317)
	at hudson.ExtensionList.iterator(ExtensionList.java:172)
	at hudson.model.User.allocateDefaultPropertyInstancesAsNeeded(User.java:206)
	at hudson.model.User.load(User.java:195)
	at hudson.model.User.<init>(User.java:188)
	at hudson.model.User.getOrCreateById(User.java:509)
	at hudson.model.User.getById(User.java:601)
	at hudson.model.User.get(User.java:586)
	at hudson.plugins.jobConfigHistory.PluginUtils.getSystemHistoryDao(PluginUtils.java:132)
	at hudson.plugins.jobConfigHistory.JobConfigHistorySaveableListener.getHistoryDao(JobConfigHistorySaveableListener.java:92)
	at hudson.plugins.jobConfigHistory.JobConfigHistorySaveableListener.onChange(JobConfigHistorySaveableListener.java:56)
	at hudson.model.listeners.SaveableListener.fireOnChange(SaveableListener.java:81)
	at hudson.model.Descriptor.save(Descriptor.java:877)
	at jenkins.telemetry.Correlator.<init>(Correlator.java:52)
	at jenkins.telemetry.Correlator$$FastClassByGuice$$5678e8d6.newInstance(<generated>)
	at com.google.inject.internal.cglib.reflect.$FastConstructor.newInstance(FastConstructor.java:40)
	at com.google.inject.internal.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:61)
	at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:105)
	at com.google.inject.internal.ConstructorInjector.access$000(ConstructorInjector.java:32)
	at com.google.inject.internal.ConstructorInjector$1.call(ConstructorInjector.java:89)
	at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:115)
	at hudson.ExtensionFinder$GuiceFinder$SezpozModule.onProvision(ExtensionFinder.java:567)
	at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:126)
	at com.google.inject.internal.ProvisionListenerStackCallback.provision(ProvisionListenerStackCallback.java:68)
	at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:87)
	at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:267)
	at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
	at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1103)
	at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
	at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:145)
	at hudson.ExtensionFinder$GuiceFinder$FaultTolerantScope$1.get(ExtensionFinder.java:439)
	at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41)
	at com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1016)
	at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092)
	at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1012)
	at hudson.ExtensionFinder$GuiceFinder._find(ExtensionFinder.java:401)
	at hudson.ExtensionFinder$GuiceFinder.find(ExtensionFinder.java:392)
	at hudson.ClassicPluginStrategy.findComponents(ClassicPluginStrategy.java:344)
	at hudson.ExtensionList.load(ExtensionList.java:381)
	at hudson.ExtensionList.ensureLoaded(ExtensionList.java:317)
	at hudson.ExtensionList.getComponents(ExtensionList.java:183)
	at hudson.DescriptorExtensionList.load(DescriptorExtensionList.java:193)
	at hudson.ExtensionList.ensureLoaded(ExtensionList.java:317)
	at hudson.ExtensionList.iterator(ExtensionList.java:172)
	at jenkins.branch.CustomOrganizationFolderDescriptor.doAddSpecificDescriptors(CustomOrganizationFolderDescriptor.java:166)
	at jenkins.branch.CustomOrganizationFolderDescriptor.addSpecificDescriptors(CustomOrganizationFolderDescriptor.java:145)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:104)
	at hudson.init.TaskMethodFinder$TaskImpl.run(TaskMethodFinder.java:175)
	at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296)
	at jenkins.model.Jenkins$5.runTask(Jenkins.java:1124)
	at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214)
	at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
2020-08-31 06:37:58.522+0000 [id=26]	INFO	h.ExtensionFinder$GuiceFinder$FaultTolerantScope$1#error: Failed to instantiate optional component hudson.plugins.build_timeout.operations.AbortAndRestartOperation$DescriptorImpl; skipping
2020-08-31 06:38:00.026+0000 [id=26]	INFO	h.ExtensionFinder$GuiceFinder$FaultTolerantScope$1#error: Failed to instantiate optional component org.biouno.unochoice.model.ScriptlerScript$DescriptorImpl; skipping
2020-08-31 06:38:00.733+0000 [id=26]	INFO	h.ExtensionFinder$GuiceFinder$FaultTolerantScope$1#error: Failed to instantiate optional component hudson.plugins.build_timeout.operations.AbortAndRestartOperation$DescriptorImpl; skipping
2020-08-31 06:38:00.735+0000 [id=26]	INFO	h.ExtensionFinder$GuiceFinder$FaultTolerantScope$1#error: Failed to instantiate optional component org.biouno.unochoice.model.ScriptlerScript$DescriptorImpl; skipping
2020-08-31 06:38:00.819+0000 [id=25]	INFO	jenkins.InitReactorRunner$1#onAttained: Augmented all extensions
2020-08-31 06:38:00.943+0000 [id=25]	INFO	jenkins.InitReactorRunner$1#onAttained: Loaded all jobs
2020-08-31 06:38:01.127+0000 [id=42]	INFO	hudson.model.AsyncPeriodicWork$1#run: Started Download metadata
2020-08-31 06:38:01.134+0000 [id=42]	INFO	hudson.util.Retrier#start: Attempt #1 to do the action check updates server
2020-08-31 06:38:02.449+0000 [id=26]	WARNING	o.j.r.u.AnonymousClassWarnings#warn: Attempt to (de-)serialize anonymous class com.gitee.jenkins.gitee.api.impl.GiteeV5ClientBuilder$1 in file:/var/jenkins_home/plugins/gitee/WEB-INF/lib/gitee-1.1.3.jar; see: https://jenkins.io/redirect/serialization-of-anonymous-classes/
2020-08-31 06:38:02.747+0000 [id=25]	INFO	o.s.c.s.AbstractApplicationContext#prepareRefresh: Refreshing org.springframework.web.context.support.StaticWebApplicationContext@c553191: display name [Root WebApplicationContext]; startup date [Mon Aug 31 14:38:02 CST 2020]; root of context hierarchy
2020-08-31 06:38:02.747+0000 [id=25]	INFO	o.s.c.s.AbstractApplicationContext#obtainFreshBeanFactory: Bean factory for application context [org.springframework.web.context.support.StaticWebApplicationContext@c553191]: org.springframework.beans.factory.support.DefaultListableBeanFactory@263fd32b
2020-08-31 06:38:02.761+0000 [id=25]	INFO	o.s.b.f.s.DefaultListableBeanFactory#preInstantiateSingletons: Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@263fd32b: defining beans [authenticationManager]; root of factory hierarchy
2020-08-31 06:38:03.224+0000 [id=25]	INFO	o.s.c.s.AbstractApplicationContext#prepareRefresh: Refreshing org.springframework.web.context.support.StaticWebApplicationContext@1c077129: display name [Root WebApplicationContext]; startup date [Mon Aug 31 14:38:03 CST 2020]; root of context hierarchy
2020-08-31 06:38:03.224+0000 [id=25]	INFO	o.s.c.s.AbstractApplicationContext#obtainFreshBeanFactory: Bean factory for application context [org.springframework.web.context.support.StaticWebApplicationContext@1c077129]: org.springframework.beans.factory.support.DefaultListableBeanFactory@214772f9
2020-08-31 06:38:03.225+0000 [id=25]	INFO	o.s.b.f.s.DefaultListableBeanFactory#preInstantiateSingletons: Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@214772f9: defining beans [filter,legacy]; root of factory hierarchy
2020-08-31 06:38:03.958+0000 [id=25]	INFO	jenkins.install.SetupWizard#init:

*************************************************************
*************************************************************
*************************************************************

Jenkins initial setup is required. An admin user has been created and a password generated.
Please use the following password to proceed to installation:

73281c7d200643cc8256a225f7c49d9f

This may also be found at: /var/jenkins_home/secrets/initialAdminPassword

*************************************************************
*************************************************************
*************************************************************

^C
[root@op-k8s-m001 jenkins]#
[root@op-k8s-m001 jenkins]#
[root@op-k8s-m001 jenkins]# kubectl describe pod jenkins-6db584c4cd-lfl9g -n devops
Name:               jenkins-6db584c4cd-lfl9g
Namespace:          devops
Priority:           0
PriorityClassName:  <none>
Node:               op-k8s-n002/192.168.1.205
Start Time:         Mon, 31 Aug 2020 14:16:21 +0800
Labels:             app=jenkins
                    pod-template-hash=6db584c4cd
Annotations:        <none>
Status:             Running
IP:                 10.244.2.208
Controlled By:      ReplicaSet/jenkins-6db584c4cd
Containers:
  jenkins:
    Container ID:   docker://54b927fc35395df798bf5983abdcd3bb4db337b37497d25910c1cc97489e7174
    Image:          registry.cn-shanghai.aliyuncs.com/yjk-datag/jenkins:2.199
    Image ID:       docker-pullable://registry.cn-shanghai.aliyuncs.com/yjk-datag/jenkins@sha256:7ab00ec2a9601b9261e50ed294b75f9a0ccfa2750070bd37b31ec80be3ece887
    Ports:          8080/TCP, 50000/TCP
    Host Ports:     0/TCP, 0/TCP
    State:          Running
      Started:      Mon, 31 Aug 2020 14:37:20 +0800
    Last State:     Terminated
      Reason:       Error
      Exit Code:    143
      Started:      Mon, 31 Aug 2020 14:34:21 +0800
      Finished:     Mon, 31 Aug 2020 14:37:20 +0800
    Ready:          False
    Restart Count:  7
    Limits:
      cpu:     1
      memory:  1Gi
    Requests:
      cpu:      500m
      memory:   1Gi
    Liveness:   http-get http://:8080/login delay=60s timeout=5s period=10s #success=1 #failure=12
    Readiness:  http-get http://:8080/login delay=60s timeout=5s period=10s #success=1 #failure=12
    Environment:
      JAVA_OPTS:  -XshowSettings:vm -Dhudson.slaves.NodeProvisioner.initialDelay=0 -Dhudson.slaves.NodeProvisioner.MARGIN=50 -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85 -Duser.timezone=Asia/Shanghai
    Mounts:
      /var/jenkins_home/plugins from pluginshome (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from jenkins-sa-token-p4rdt (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  test-jenkinshome:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  test-pvc
    ReadOnly:   false
  m2home:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  m2-pvc
    ReadOnly:   false
  pluginshome:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  jenkins-plugins-pvc
    ReadOnly:   false
  jenkins-sa-token-p4rdt:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  jenkins-sa-token-p4rdt
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason     Age                  From                  Message
  ----     ------     ----                 ----                  -------
  Normal   Scheduled  22m                  default-scheduler     Successfully assigned devops/jenkins-6db584c4cd-lfl9g to op-k8s-n002
  Normal   Pulled     22m                  kubelet, op-k8s-n002  Container image "registry.cn-shanghai.aliyuncs.com/yjk-datag/jenkins:2.199" already present on machine
  Normal   Created    22m                  kubelet, op-k8s-n002  Created container jenkins
  Normal   Started    22m                  kubelet, op-k8s-n002  Started container jenkins
  Warning  Unhealthy  19m (x11 over 21m)   kubelet, op-k8s-n002  Liveness probe failed: HTTP probe failed with statuscode: 503
  Warning  Unhealthy  2m8s (x79 over 21m)  kubelet, op-k8s-n002  Readiness probe failed: HTTP probe failed with statuscode: 503
[root@op-k8s-m001 jenkins]# kubectl get pods -n devops -o wide -w
NAME                       READY   STATUS             RESTARTS   AGE   IP             NODE          NOMINATED NODE   READINESS GATES
jenkins-6db584c4cd-lfl9g   0/1     CrashLoopBackOff   7          24m   10.244.2.208   op-k8s-n002   <none>           <none>



^C[root@op-k8s-m001 jenkins]# kubectl describe pod jenkins-6db584c4cd-lfl9g -n devops
Name:               jenkins-6db584c4cd-lfl9g
Namespace:          devops
Priority:           0
PriorityClassName:  <none>
Node:               op-k8s-n002/192.168.1.205
Start Time:         Mon, 31 Aug 2020 14:16:21 +0800
Labels:             app=jenkins
                    pod-template-hash=6db584c4cd
Annotations:        <none>
Status:             Running
IP:                 10.244.2.208
Controlled By:      ReplicaSet/jenkins-6db584c4cd
Containers:
  jenkins:
    Container ID:   docker://54b927fc35395df798bf5983abdcd3bb4db337b37497d25910c1cc97489e7174
    Image:          registry.cn-shanghai.aliyuncs.com/yjk-datag/jenkins:2.199
    Image ID:       docker-pullable://registry.cn-shanghai.aliyuncs.com/yjk-datag/jenkins@sha256:7ab00ec2a9601b9261e50ed294b75f9a0ccfa2750070bd37b31ec80be3ece887
    Ports:          8080/TCP, 50000/TCP
    Host Ports:     0/TCP, 0/TCP
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Error
      Exit Code:    143
      Started:      Mon, 31 Aug 2020 14:37:20 +0800
      Finished:     Mon, 31 Aug 2020 14:40:20 +0800
    Ready:          False
    Restart Count:  7
    Limits:
      cpu:     1
      memory:  1Gi
    Requests:
      cpu:      500m
      memory:   1Gi
    Liveness:   http-get http://:8080/login delay=60s timeout=5s period=10s #success=1 #failure=12
    Readiness:  http-get http://:8080/login delay=60s timeout=5s period=10s #success=1 #failure=12
    Environment:
      JAVA_OPTS:  -XshowSettings:vm -Dhudson.slaves.NodeProvisioner.initialDelay=0 -Dhudson.slaves.NodeProvisioner.MARGIN=50 -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85 -Duser.timezone=Asia/Shanghai
    Mounts:
      /var/jenkins_home/plugins from pluginshome (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from jenkins-sa-token-p4rdt (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  test-jenkinshome:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  test-pvc
    ReadOnly:   false
  m2home:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  m2-pvc
    ReadOnly:   false
  pluginshome:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  jenkins-plugins-pvc
    ReadOnly:   false
  jenkins-sa-token-p4rdt:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  jenkins-sa-token-p4rdt
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason     Age                   From                  Message
  ----     ------     ----                  ----                  -------
  Normal   Scheduled  24m                   default-scheduler     Successfully assigned devops/jenkins-6db584c4cd-lfl9g to op-k8s-n002
  Normal   Pulled     24m                   kubelet, op-k8s-n002  Container image "registry.cn-shanghai.aliyuncs.com/yjk-datag/jenkins:2.199" already present on machine
  Normal   Created    24m                   kubelet, op-k8s-n002  Created container jenkins
  Normal   Started    24m                   kubelet, op-k8s-n002  Started container jenkins
  Warning  Unhealthy  21m (x11 over 23m)    kubelet, op-k8s-n002  Liveness probe failed: HTTP probe failed with statuscode: 503
  Warning  Unhealthy  4m30s (x79 over 23m)  kubelet, op-k8s-n002  Readiness probe failed: HTTP probe failed with statuscode: 503
[root@op-k8s-m001 jenkins]#

原来官方镜像jenkins/jenkins:2.199启动时间参考 每次都会重启至少一次 启动时间5m以上，有时候甚至起不来一直卡在503

现在新镜像时间参考 大小的话原先300M不到 我新作的500M不到。大小有差别 但是个人比较喜欢用centos

健康检查通过比较快 不过我后来是注释掉了readiness和liveness

2 创建pvc rbac deploy的yaml文件如下

首先将需要持久化存储 共享的文件通过nfs 挂载出来 我这里是选择192.168.1.171

maven 仓库 一些jenkins的文件夹(如插件 工作路径等，不然我每次都要重新下插件 下载代码比较耗时也比较麻烦) 我拿/opt/workspace是来测试

jenkins-pvc.yaml

---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: jenkins-pv
  namespace: devops
spec:
  capacity:
    storage: 200Gi
  accessModes:
  - ReadWriteMany
  persistentVolumeReclaimPolicy: Delete
  nfs:
    server: 192.168.1.171
    path: /root/.jenkins


---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: jenkins-pvc
  namespace: devops
spec:
  accessModes:
  - ReadWriteMany
  resources:
    requests:
      storage: 200Gi

---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: test-pv
  namespace: devops
spec:
  capacity:
    storage: 50Gi
  accessModes:
  - ReadWriteMany
  persistentVolumeReclaimPolicy: Delete
  nfs:
    server: 192.168.1.171
    path: /opt/workspace


---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: test-pvc
  namespace: devops
spec:
  accessModes:
  - ReadWriteMany
  resources:
    requests:
      storage: 50Gi

---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: m2-pv
  namespace: devops
spec:
  capacity:
    storage: 200Gi
  accessModes:
  - ReadWriteMany
  persistentVolumeReclaimPolicy: Delete
  nfs:
    server: 192.168.1.171
    path: /srv/m2


---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: m2-pvc
  namespace: devops
spec:
  accessModes:
  - ReadWriteMany
  resources:
    requests:
      storage: 200Gi


---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: jenkins-plugins-pv
  namespace: devops
spec:
  capacity:
    storage: 200Gi
  accessModes:
  - ReadWriteMany
  persistentVolumeReclaimPolicy: Delete
  nfs:
    server: 192.168.1.171
    path: /root/.jenkins/plugins


---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: jenkins-plugins-pvc
  namespace: devops
spec:
  accessModes:
  - ReadWriteMany
  resources:
    requests:
      storage: 200Gi

jenkins-rbac.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: jenkins-sa
  namespace: devops

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: jenkins-cr
rules:
  - apiGroups: ["extensions", "apps"]
    resources: ["deployments"]
    verbs: ["create", "delete", "get", "list", "watch", "patch", "update"]
  - apiGroups: [""]
    resources: ["services"]
    verbs: ["create", "delete", "get", "list", "watch", "patch", "update"]
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["create","delete","get","list","patch","update","watch"]
  - apiGroups: [""]
    resources: ["pods/exec"]
    verbs: ["create","delete","get","list","patch","update","watch"]
  - apiGroups: [""]
    resources: ["pods/log"]
    verbs: ["get","list","watch"]
  - apiGroups: [""]
    resources: ["secrets"]
    verbs: ["get"]

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: jenkins-crd
roleRef:
  kind: ClusterRole
  name: jenkins-cr
  apiGroup: rbac.authorization.k8s.io
subjects:
- kind: ServiceAccount
  name: jenkins-sa
  namespace: devops

test-jenkins-deploy.yaml

---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: jenkins
  namespace: devops
spec:
  template:
    metadata:
      labels:
        app: jenkins
    spec:
      terminationGracePeriodSeconds: 10
      serviceAccount: jenkins-sa
      containers:
      - name: jenkins
        image:  registry.cn-shanghai.aliyuncs.com/yjk-datag/jenkins:master0406
        #image: registry.cn-shanghai.aliyuncs.com/yjk-datag/jenkins:2.199base01
        env:
        - name: JAVA_OPTS
          value: -XshowSettings:vm -Dhudson.slaves.NodeProvisioner.initialDelay=0 -Dhudson.slaves.NodeProvisioner.MARGIN=50 -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85 -Duser.timezone=Asia/Shanghai
        ports:
        - containerPort: 8080
          name: web
          protocol: TCP
        - containerPort: 50000
          name: agent
          protocol: TCP
        resources:
          limits:
            cpu: 2000m
            memory: 2Gi
          requests:
            cpu: 1000m
            memory: 1Gi
#        livenessProbe:
#          httpGet:
#            path: /jenkins/login
#            port: 8080
#          initialDelaySeconds: 60
#          timeoutSeconds: 5
#          failureThreshold: 12
#        readinessProbe:
#          httpGet:
#            path: /jenkins/login
#            port: 8080
#          initialDelaySeconds: 60
#          timeoutSeconds: 5
#          failureThreshold: 12
        volumeMounts:
        - name: test-jenkinshome
          mountPath: /root/.jenkins/workspace
        - name: pluginshome
          mountPath: /root/.jenkins/plugins
        - name: m2home
          mountPath: /srv/m2
      securityContext:
        fsGroup: 1000
      volumes:
      - name: test-jenkinshome
        persistentVolumeClaim:
          claimName: test-pvc
      - name: m2home
        persistentVolumeClaim:
          claimName: m2-pvc
      - name: pluginshome
        persistentVolumeClaim:
          claimName: jenkins-plugins-pvc

      imagePullSecrets:
      - name: aliyun-ns-devops

---
apiVersion: v1
kind: Service
metadata:
  name: jenkins
  namespace: devops
  labels:
    app: jenkins
spec:
  selector:
    app: jenkins
  type: NodePort
  ports:
  - name: web
    port: 8080
    targetPort: web
    nodePort: 30002
  - name: agent
    port: 50000 # 这个要和代理指定端口一致
    targetPort: agent

kubectl apply -f  jenkins-pvc.yaml
kubectl apply -f  jenkins-rbac.yaml
kubectl apply -f  test-jenkins-deploy.yaml

3 调整jenkins-master镜像

jenkins 启动慢 经常卡在jenkins正在启动 状态为503 主要需要修改2个文件/root/.jenkins/hudson.model.UpdateCenter.xml和/root/.jenkins/updates/default.json

修改这两个文件

cat > /root/.jenkins/hudson.model.UpdateCenter.xml <<EOF
<?xml version='1.1' encoding='UTF-8'?>
<sites>
  <site>
    <id>default</id>
    <url>http://mirror.xmission.com/jenkins/updates/update-center.json</url>
  </site>
</sites>
EOF

sed -i 's/www.google.com/www.baidu.com/g'  /root/.jenkins/updates/default.json

登陆jenkins配置基础项

密码通过 kubectl exec $pod_name -n devops 之后 cat /root/.jenkins/secrets/initialAdminPassword 查看

进入系统管理 > 系统配置 > 新增一个云(需要先装上kubernetes插件)

上面配置的config.xml如下

<?xml version='1.1' encoding='UTF-8'?>
<hudson>
  <disabledAdministrativeMonitors/>
  <version>2.199</version>
  <installStateName>RUNNING</installStateName>
  <numExecutors>2</numExecutors>
  <mode>NORMAL</mode>
  <useSecurity>true</useSecurity>
  <authorizationStrategy class="hudson.security.FullControlOnceLoggedInAuthorizationStrategy">
    <denyAnonymousReadAccess>true</denyAnonymousReadAccess>
  </authorizationStrategy>
  <securityRealm class="hudson.security.HudsonPrivateSecurityRealm">
    <disableSignup>true</disableSignup>
    <enableCaptcha>false</enableCaptcha>
  </securityRealm>
  <disableRememberMe>false</disableRememberMe>
  <projectNamingStrategy class="jenkins.model.ProjectNamingStrategy$DefaultProjectNamingStrategy"/>
  <workspaceDir>${JENKINS_HOME}/workspace/${ITEM_FULL_NAME}</workspaceDir>
  <buildsDir>${ITEM_ROOTDIR}/builds</buildsDir>
  <markupFormatter class="hudson.markup.EscapedMarkupFormatter"/>
  <jdks/>
  <viewsTabBar class="hudson.views.DefaultViewsTabBar"/>
  <myViewsTabBar class="hudson.views.DefaultMyViewsTabBar"/>
  <clouds>
    <org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud plugin="kubernetes@1.21.1">
      <name>kubernetes</name>
      <defaultsProviderTemplate></defaultsProviderTemplate>
      <templates>
        <org.csanchez.jenkins.plugins.kubernetes.PodTemplate>
          <inheritFrom></inheritFrom>
          <name>jenkins-slave</name>
          <namespace>devops</namespace>
          <privileged>false</privileged>
          <capOnlyOnAlivePods>false</capOnlyOnAlivePods>
          <alwaysPullImage>false</alwaysPullImage>
          <instanceCap>2147483647</instanceCap>
          <slaveConnectTimeout>100</slaveConnectTimeout>
          <idleMinutes>0</idleMinutes>
          <activeDeadlineSeconds>0</activeDeadlineSeconds>
          <label>jenkins-jnlp</label>
          <serviceAccount>jenkins-sa</serviceAccount>
          <nodeSelector></nodeSelector>
          <nodeUsageMode>EXCLUSIVE</nodeUsageMode>
          <hostNetwork>false</hostNetwork>
          <workspaceVolume class="org.csanchez.jenkins.plugins.kubernetes.volumes.workspace.EmptyDirWorkspaceVolume">
            <memory>false</memory>
          </workspaceVolume>
          <volumes>
            <org.csanchez.jenkins.plugins.kubernetes.volumes.HostPathVolume>
              <mountPath>/var/run/docker.sock</mountPath>
              <hostPath>/var/run/docker.sock</hostPath>
            </org.csanchez.jenkins.plugins.kubernetes.volumes.HostPathVolume>
            <org.csanchez.jenkins.plugins.kubernetes.volumes.HostPathVolume>
              <mountPath>/root/.kube</mountPath>
              <hostPath>/root/.kube</hostPath>
            </org.csanchez.jenkins.plugins.kubernetes.volumes.HostPathVolume>
            <org.csanchez.jenkins.plugins.kubernetes.volumes.NfsVolume>
              <mountPath>/root/.jenkins/workspace</mountPath>
              <serverAddress>192.168.1.171</serverAddress>
              <serverPath>/opt/workspace</serverPath>
              <readOnly>false</readOnly>
            </org.csanchez.jenkins.plugins.kubernetes.volumes.NfsVolume>
            <org.csanchez.jenkins.plugins.kubernetes.volumes.NfsVolume>
              <mountPath>/usr/share/maven/conf</mountPath>
              <serverAddress>192.168.1.171</serverAddress>
              <serverPath>/usr/share/maven/conf</serverPath>
              <readOnly>false</readOnly>
            </org.csanchez.jenkins.plugins.kubernetes.volumes.NfsVolume>
            <org.csanchez.jenkins.plugins.kubernetes.volumes.NfsVolume>
              <mountPath>/srv/m2</mountPath>
              <serverAddress>192.168.1.171</serverAddress>
              <serverPath>/srv/m2</serverPath>
              <readOnly>false</readOnly>
            </org.csanchez.jenkins.plugins.kubernetes.volumes.NfsVolume>
          </volumes>
          <containers>
            <org.csanchez.jenkins.plugins.kubernetes.ContainerTemplate>
              <name>jnlp</name>
              <image>registry.cn-shanghai.aliyuncs.com/yjk-datag/jenkins:jnlp6.09</image>
              <privileged>false</privileged>
              <alwaysPullImage>false</alwaysPullImage>
              <workingDir>/root/.jenkins</workingDir>
              <command></command>
              <args></args>
              <ttyEnabled>true</ttyEnabled>
              <resourceRequestCpu></resourceRequestCpu>
              <resourceRequestMemory></resourceRequestMemory>
              <resourceLimitCpu></resourceLimitCpu>
              <resourceLimitMemory></resourceLimitMemory>
              <envVars/>
              <ports/>
              <livenessProbe>
                <execArgs></execArgs>
                <timeoutSeconds>0</timeoutSeconds>
                <initialDelaySeconds>0</initialDelaySeconds>
                <failureThreshold>0</failureThreshold>
                <periodSeconds>0</periodSeconds>
                <successThreshold>0</successThreshold>
              </livenessProbe>
            </org.csanchez.jenkins.plugins.kubernetes.ContainerTemplate>
          </containers>
          <envVars/>
          <annotations/>
          <imagePullSecrets>
            <org.csanchez.jenkins.plugins.kubernetes.PodImagePullSecret>
              <name>aliyun-ns-devops</name>
            </org.csanchez.jenkins.plugins.kubernetes.PodImagePullSecret>
          </imagePullSecrets>
          <nodeProperties/>
          <yamlMergeStrategy class="org.csanchez.jenkins.plugins.kubernetes.pod.yaml.Overrides"/>
          <showRawYaml>true</showRawYaml>
          <podRetention class="org.csanchez.jenkins.plugins.kubernetes.pod.retention.Default"/>
        </org.csanchez.jenkins.plugins.kubernetes.PodTemplate>
      </templates>
      <serverUrl>https://kubernetes.default.svc.cluster.local</serverUrl>
      <skipTlsVerify>false</skipTlsVerify>
      <addMasterProxyEnvVars>false</addMasterProxyEnvVars>
      <capOnlyOnAlivePods>false</capOnlyOnAlivePods>
      <namespace>devops</namespace>
      <directConnection>false</directConnection>
      <jenkinsUrl>http://jenkins.devops.svc.cluster.local:8080</jenkinsUrl>
      <containerCap>10</containerCap>
      <retentionTimeout>5</retentionTimeout>
      <connectTimeout>5</connectTimeout>
      <readTimeout>15</readTimeout>
      <podLabels>
        <org.csanchez.jenkins.plugins.kubernetes.PodLabel>
          <key>jenkins</key>
          <value>slave</value>
        </org.csanchez.jenkins.plugins.kubernetes.PodLabel>
      </podLabels>
      <usageRestricted>false</usageRestricted>
      <maxRequestsPerHost>32</maxRequestsPerHost>
      <waitForPodSec>600</waitForPodSec>
      <podRetention class="org.csanchez.jenkins.plugins.kubernetes.pod.retention.Never"/>
    </org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud>
  </clouds>
  <quietPeriod>5</quietPeriod>
  <scmCheckoutRetryCount>0</scmCheckoutRetryCount>
  <views>
    <hudson.model.AllView>
      <owner class="hudson" reference="../../.."/>
      <name>all</name>
      <filterExecutors>false</filterExecutors>
      <filterQueue>false</filterQueue>
      <properties class="hudson.model.View$PropertyList"/>
    </hudson.model.AllView>
  </views>
  <primaryView>all</primaryView>
  <slaveAgentPort>50000</slaveAgentPort>
  <label></label>
  <crumbIssuer class="hudson.security.csrf.DefaultCrumbIssuer">
    <excludeClientIPFromCrumb>false</excludeClientIPFromCrumb>
  </crumbIssuer>
  <nodeProperties/>
  <globalNodeProperties/>
</hudson>

修改 系统管理 > 全局安全配置 > 代理 修改端口为50000

不改端口也会导致slave agent 起不来

调整完毕 执行commit

 docker commit -a "build success" be59cef93de7    registry.cn-shanghai.aliyuncs.com/yjk-datag/jenkins:master0407
 docker push registry.cn-shanghai.aliyuncs.com/yjk-datag/jenkins:master0407

然后修改test-jenkins-deploy.yaml的镜像为最新镜像

# 应用新的master镜像
kubectl apply -f test-jenkins-deploy.yaml --force

4 创建jenkins-slave镜像

因为slave官方镜像commit之后 无法和master连接 就自己做了镜像

slave镜像不支持中文 可以通过env添加

准备的东西如下

1 Dockerfile

2 java解压好的文件夹(上面的jdk安装包或者别的机子上复制过来都可以)

3 jenkins-slave

4 libltdl.so.7

5 slave.jar

6 .ssh 文件 用于下载ssh 的git代码 以及config文件 用于避免下载代码的时候交互yea/no 否则可能会卡在git clone

Dockerfile 如下

FROM openshift/base-centos7
# FROM centos:centos7.4.1708 换成这个应该可以支持中文
MAINTAINER PDABC Enterprise Container Images <jiaminxu@hz-health.cn>
# COPY maven /usr/share/maven
COPY java       /usr/local/java
COPY libltdl.so.7 /usr/lib64/libltdl.so.7
COPY slave.jar /usr/share/jenkins/slave.jar
COPY jenkins-slave /usr/local/bin/jenkins-slave
COPY .ssh /root/.ssh

# ENV LANG=en_US.UTF-8
# ENV LANGUAGE=en_US.UTF-8
ENV HOME /root/.jenkins
ENV AGENT_WORKDIR=/root/.jenkins/agent
ENV JAVA_HOME /usr/local/java
ENV MAVEN_HOME /usr/local/maven/
ENV CLASSPATH .:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
ENV PATH ${JAVA_HOME}/bin:${MAVEN_HOME}/bin:${PATH}
ENV MAVEN_CONFIG "$USER_HOME_DIR/.m2"

RUN  chmod 755 /usr/share/jenkins \
  && chmod 644 /usr/share/jenkins/slave.jar

RUN mkdir -p /root/.jenkins \
 && mkdir -p ${AGENT_WORKDIR} \
 && yum -y install git subversion sshpassi maven

VOLUME /root/.jenkins
VOLUME ${AGENT_WORKDIR}
WORKDIR /root/.jenkins

USER root
ENTRYPOINT ["jenkins-slave"]

build 上传

docker build -t registry.cn-shanghai.aliyuncs.com/yjk-datag/jenkins:jnlp6.09 .
docker push registry.cn-shanghai.aliyuncs.com/yjk-datag/jenkins:jnlp6.09 

5 创建一个job或者scp一个job 到jenkins中

# 我这里比较偷懒 直接scp了一个过来
scp -r 192.168.1.171:/root/.jenkins/jobs/test-yjk-k8s /root/.jenkins/jobs/

然后reload jenkins

http://{node_ip}:Nodeport/reload

reload之后 scp过来的job会加载出来

job的pipeline配置如下

这里需要修改node为node(‘jenkins-jnlp’) ，带上标签，而且我注释了部署脚本，只执行了agent编译。后期需要部署 还需要nfs挂载部署脚本的文件夹。

// Using git without checkout 
pipeline {
  agent any
  parameters {
    gitParameter branchFilter: 'origin/(.*)', defaultValue: 'master', name: 'Branch', sortMode: 'DESCENDING_SMART', type: 'PT_BRANCH'
  }

  stages {
        // stage("deleteDir") {
        //     steps{
        //         script{
        //             println env.WORKSPACE
        //             //dir("${env.WORKSPACE}/testdata"){   //切换到当前工作目录下的testdata目录
        //             //   sh "pwd"                         //sh命令可以 sh "command..." 也可以 sh("command....")
        //             }
        //             sh("ls -al ${env.WORKSPACE}")
        //             deleteDir()  // clean up current work directory   //清空目录
        //             sh("ls -al ${env.WORKSPACE}")
        //         }
        //     }
    stage('更新拉取最新代码') {
      steps {
        git branch: "${params.Branch}", url: 'git@gitlab.ihaozhuo.com:Java_Service/YJK-Java.git'
      }
    }
    stage('sonar扫描') {
      steps {
        sh "echo ${WORKSPACE} "
        sh "echo ${JOB_NAME} "
        //sh "echo ${deploy_step} "
        sh "echo ${env.JOB_NAME} "
        script
        {
            switch(SONAR_SCAN)
            {
                case "TRUE":
                    sh "echo 扫描"
                    sh "sh  /srv/deploy_scripts/do_sonar.sh ${WORKSPACE} ${JOB_NAME}"
                    // sh "echo \u001B[31m不扫描\u001B[0m ！！！"
                default:
                    sh "echo \u001B[31m不扫描\u001B[0m ！！！"
                    // sh "sh  /srv/deploy_scripts/do_sonar.sh ${WORKSPACE} ${JOB_NAME}"
            
        }
        }
            }

      }
    stage('输出项目的参数') {
      steps {
        sh "echo ${WORKSPACE} "
        sh "echo ${Branch} "
        //sh "echo ${deploy_step} "
        sh "echo ${env.JOB_NAME} "
        script {
            wrap([$class: 'BuildUser']) {
            //发布war包到指定服务器，虚拟机文件目录通过shell脚本初始化建立，所以目录是固定的
            //sh "sshpass -p ${serverPasswd} scp ${params.warLocation} ${serverName}@${serverIP}:htdocs/war"
            //这里增加了一个小功能，在服务器上记录了基本部署信息，方便多人使用一套环境时问题排查，storge in {WORKSPACE}/deploy.log  & remoteServer:htdocs/war
            //Date date = new Date()
            def deploylog="${BUILD_USER} use pipeline  '${JOB_NAME}(${BUILD_NUMBER})' "
            println deploylog
            //sh "echo ${deploylog} >>${WORKSPACE}/deploy.log"
            //sh "sshpass -p ${serverPasswd} scp ${WORKSPACE}/deploy.log ${serverName}@${serverIP}:htdocs/war"
            //jetty restart，重启jetty
            //sh "sshpass -p ${serverPasswd} ssh ${serverName}@${serverIP} 'bin/jettyrestart.sh' "
            buildName "#${BUILD_NUMBER}-^${BRANCH}^-${BUILD_USER}"
            HTTPD_LOCATION= sh(  returnStdout: true, script: 'git show -s  |grep -vE "commit|Date" |grep -v "^$" |sed "s/^[ \\t]*//g"')
            //echo "HTTPD_LOCATION =${HTTPD_LOCATION}"
            //def color_model="<font color=\"red\">$Model </font>"
            
            buildDescription  "${HTTPD_LOCATION} ${BUILD_USER}此次部署项目: ${Model} " 
            }
        }
        script
        {
            switch(Model)
            {
                case "all":
                    sh "echo \u001B[31m部署全部模块\u001B[0m ！！！"
                default:
                    sh "echo others"
            
        }
        }
            }

      }
    }

  }

// 这个暂时没用   
def doOneProject(p_name){
   stage('当前执行工程') {
     echo "${p_name}"
   }
   stage('工程执行结果') {
      echo "结束了${p_name}"
   }
}

node('jenkins-jnlp') {
    timestamps {
    env.PRO_TYPE = "test"
    ansiColor('xterm') {
    stage('Java编译') { 
        sh "echo ${WORKSPACE} "
        sh "echo ${Branch} "
        //sh "echo ${deploy_step} "
        sh "echo ${env.JOB_NAME} "
        script
        {
            switch(Model)
            {
                case "yjk-all":
                    sh "echo \u001B[31m部署全部模块\u001B[0m ！！！"
                    sh "cd ${WORKSPACE} && rm dev-properties/ -rf && git clone git@gitlab.ihaozhuo.com:dev-properties/dev-properties.git  &&  cd ${WORKSPACE}/haozhuo/ && mvn clean install -T 1C -Dmaven.test.skip=true  -Dmaven.compile.fork=true -Ptest -Dautoconfig.userProperties=${WORKSPACE}/dev-properties/test1.properties -Dautoconfig.charset=utf-8 -Dautoconfig.interactive=off"
                default:
                    sh "echo 部署  \u001B[32m $Model \u001B[0m !!!"
                    sh "cd ${WORKSPACE} && rm dev-properties/ -rf && git clone git@gitlab.ihaozhuo.com:dev-properties/dev-properties.git  &&  cd ${WORKSPACE}/haozhuo/ && mvn clean install  -pl $model -am -U  -T 1C -Dmaven.test.skip=true  -Dmaven.compile.fork=true -Ptest -Dautoconfig.userProperties=${WORKSPACE}/dev-properties/test1.properties -Dautoconfig.charset=utf-8 -Dautoconfig.interactive=off"
            
        }
        }
   }
   stage('执行数据库脚本') { 
        // sh "/bin/bash /srv/yaml/autosql.sh  redis  $version"
        // sh "/bin/bash /srv/yaml/autosql.sh  mysql  $version"
        // sh "/bin/bash /srv/yaml/autosql.sh  mongodb  $version"


   }
    stage('统一构建镜像并发布') { 
        sh "echo ${Model}"
        sh "echo ${WORKSPACE} "
        sh "echo ${env.PRO_TYPE} "
        //sh "echo ${deploy_step} "
        sh "echo ${BUILD_NUMBER} "
        sh "echo sh /srv/deploy_scripts/k8s_deploy.sh $Model $WORKSPACE ${env.PRO_TYPE}  $BUILD_NUMBER ${env.PRO_TYPE}"
        sh "echo build successful"

   }


}
}

}

6 构建job测试及结果

因为挂载了m2仓库 所以下载依赖会比较少 部署比较快

至此部署成功 如果有需要 可以再commit一次。