golang微服务项目通用流水线

golang微服务项目通用流水线

工作中随着业务越来越大,微服务的项目也越来越多,最开始的时候是一个服务一个流水线,然后还分了三个环境,也就是一个服务三个流水线,后面就越来越不利于管理维护了,因此,进行了一个改造整合

golang这边的微服务基本分成三类,service、schedule、mq,所以将所有流水线整合为三条通用流水线,再通过参数传递的形式,去构建每个服务

一、dockerfile和jenkinsfile

这是service里面的dockerfile和jenkinsfile,schedule、mq大同小异

dockerfile

FROM ccr.ccs.tencentyun.com/chens/golang:1.17.13
ENV ZBE_PATH /biz-code
ARG SERVICENAME=message
COPY $SERVICENAME /biz-code/$SERVICENAME
WORKDIR /biz-code
RUN chmod 777 $(echo `ls -l` |awk -F ' ' '{print $NF}')
USER root
RUN ls -l
CMD ./$(echo `ls -l` |awk -F ' ' '{print $NF}')

jenkinsfile

pipeline {
  agent {
    kubernetes {cloud 'kubernetes'
      cloud 'kubernetes-test'
      slaveConnectTimeout 1200
      workspaceVolume hostPathWorkspaceVolume(hostPath:"/opt/workspace", readOnly: false)
      yaml '''
apiVersion: v1
kind: Pod
spec:
  containers:
    - args: [\'$(JENKINS_SECRET)\', \'$(JENKINS_NAME)\']
      image: 'jenkins/jnlp-slave:latest-jdk11'
      name: jnlp
      imagePullPolicy: IfNotPresent
      volumeMounts:
        - mountPath: "/etc/localtime"
          name: "localtime"
          readOnly: false 
    - command:
        - "cat"
      env:
        - name: "LANGUAGE"
          value: "en_US:en"
        - name: "LC_ALL"
          value: "en_US.UTF-8"
        - name: "LANG"
          value: "en_US.UTF-8"
      image: "ccr.ccs.tencentyun.com/chens/golang:1.17.13"
      imagePullPolicy: "IfNotPresent"
      name: "build"
      tty: true
      volumeMounts:
        - mountPath: "/etc/localtime"
          name: "localtime"
        - mountPath: "/go/pkg/"
          name: "cachedir"
          readOnly: false
    - command:
        - "cat"
      env:
        - name: "LANGUAGE"
          value: "en_US:en"
        - name: "LC_ALL"
          value: "en_US.UTF-8"
        - name: "LANG"
          value: "en_US.UTF-8"
      image: "registry.cn-beijing.aliyuncs.com/citools/kubectl:self-1.17"
      imagePullPolicy: "IfNotPresent"
      name: "kubectl"
      tty: true
      volumeMounts:
        - mountPath: "/etc/localtime"
          name: "localtime"
          readOnly: false
    - command:
        - "cat"
      env:
        - name: "LANGUAGE"
          value: "en_US:en"
        - name: "LC_ALL"
          value: "en_US.UTF-8"
        - name: "LANG"
          value: "en_US.UTF-8"
      image: "registry.cn-beijing.aliyuncs.com/citools/docker:19.03.9-git"
      imagePullPolicy: "IfNotPresent"
      name: "docker"
      tty: true
      volumeMounts:
        - mountPath: "/etc/localtime"
          name: "localtime"
          readOnly: false
        - mountPath: "/var/run/docker.sock"
          name: "dockersock"
          readOnly: false
  restartPolicy: "Never"
  imagePullSecrets:
  - name: qcloudregistrykey
  nodeSelector:
    build: "true"
  securityContext: {}
  volumes:
    - hostPath:
        path: "/var/run/docker.sock"
      name: "dockersock"
    - hostPath:
        path: "/usr/share/zoneinfo/Asia/Shanghai"
      name: "localtime"
    - name: "cachedir"
      hostPath:
        path: "/opt/gopkg"
    - name: "phpdir"
      hostPath:
        path: "/opt/phppkg"
'''
   }
}
stages {
  stage('Pulling Code') {
	parallel {
	  stage('Pulling Code by Jenkins') {
		when {
		  expression {
			env.giteeBranch == null
		  }
		}
		steps {
		  git(changelog: true, poll: true, url:'https://gitee.com/chens-go/kobe.git', branch:"${BRANCH}", credentialsId: 'gitee-mima')
   		  script {
   		    COMMIT_ID = sh(returnStdout: true, script: "git log -n 1 --pretty=format:'%h'").trim()
   		    TAG = BUILD_TAG + '-' + COMMIT_ID
   		    println "Current branch is ${BRANCH}, Commit ID is ${COMMIT_ID}, Image TAG is ${TAG}"

		  }
		}
     }
	  stage('Pulling Code by trigger') {
		when {
		  expression {
			env.giteeBranch != null
		  }
		}
		steps {
		  git(url: 'https://gitee.com/chens-go/kobe.git', branch: env.giteeBranch, changelog: true, poll: true, credentialsId: 'gitee-mima')
		  script {
			COMMIT_ID = sh(returnStdout: true, script: "git log -n 1 --pretty=format:'%h'").trim()
			TAG = BUILD_TAG + '-' + COMMIT_ID
			println "Current branch is ${env.giteeBranch}, Commit ID is ${COMMIT_ID}, Image TAG is ${TAG}"
		  }
		}
	}
  }
}
	  stage('Building') {
		steps {
		  container(name: 'build') {
			sh """ 
                go env -w GO111MODULE=on
                go env -w GOPROXY=https://goproxy.io,direct
                go env -w GOPRIVATE=gitee.com/chens-go/jordan
                export GIT_TERMINAL_PROMPT=1
                go build -o ${SERVICENAME} services/${SERVICENAME}/${SERVICENAME}-service/cmd/${SERVICENAME}/main.go
                chmod +x ${SERVICENAME}
                cp -p ${SERVICENAME} build-work/k8s2/service
			"""
		  }
		}
	  }
	  stage('Docker build for creating image') {
		environment {
		  HARBOR_USER = credentials('registry-secret')
		}
		steps {
		  container(name: 'docker') {
			sh """
              cd build-work/k8s2/service
			  echo ${HARBOR_USER_USR} ${HARBOR_USER_PSW} ${TAG}
              docker login -u ${HARBOR_USER_USR} -p ${HARBOR_USER_PSW} ${HARBOR_ADDRESS}
			  docker build -t ${HARBOR_ADDRESS}/${REGISTRY_DIR}/${IMAGE_NAME}:${TAG} --build-arg SERVICENAME=${SERVICENAME} .
			  docker push ${HARBOR_ADDRESS}/${REGISTRY_DIR}/${IMAGE_NAME}:${TAG}
			"""
		  }
		}
	  }
	  stage('Deploying to K8s') {
		environment {
		  MY_KUBECONFIG = credentials('k8s-config')
		}
		steps {
		  container(name: 'kubectl'){
			sh """
			  /usr/local/bin/kubectl --kubeconfig $MY_KUBECONFIG set image deploy -l app=${KOBE_NAME} ${KOBE_NAME}=${HARBOR_ADDRESS}/${REGISTRY_DIR}/${IMAGE_NAME}:${TAG} -n $NAMESPACE
			"""
		  }
		}
	  }
	}
  environment {
    COMMIT_ID = ""
    HARBOR_ADDRESS = "ccr.ccs.tencentyun.com"
    REGISTRY_DIR = "chens"
    IMAGE_NAME = "kobe"
    NAMESPACE = "${ENV}"
    TAG = ""
    KOBE_NAME = "${SERVICENAME}"
}
  parameters {
	gitParameter(branch: '', branchFilter: 'origin/(.*)', defaultValue: "${BRANCH}", description: 'Branch for build and deploy', name:'BRANCH', quickFilterEnabled: false, selectedValue: 'NONE', sortMode: 'NONE', tagFilter: '*', type: 'PT_BRANCH')
  }
}

二、参数传递

流水线里面需要传递三个参数

在这里插入图片描述

具体配置

在这里插入图片描述

在这里插入图片描述

在这里插入图片描述

在这里插入图片描述

在这里插入图片描述

最后的这个build-work/k8s2/service/Jenkinsfile,是jenkinsfile在项目代码中的路径

所有参数名称和路径皆可根据自己项目来定义

三、写在最后

这里面有些语法看着很离谱,其实是我在做的过程中踩的一些坑,你们可以试试不那样写会发生什么,当然也有可能还有别的解决方法,可多尝试


mq通用流水线

记录一下,因为这个流水线里面针对服务名称的格式化做了一些特殊处理
我们的mq服务都是以下划线命名,比如goods_stock_merchant,但是在k8s里面使用命令更新deploy的时候,命名不能使用下划线,所以只能在流水线里面将下划线改为中划线

Dockerfile

FROM ccr.ccs.tencentyun.com/zhubaoe/golang:1.17.13
ENV ZBE_PATH /biz-code
ARG SERVICENAME=message
COPY mq_${SERVICENAME}_server /biz-code/mq_${SERVICENAME}_server
WORKDIR /biz-code
RUN chmod 777 $(echo `ls -l` |awk -F ' ' '{print $NF}')
USER root
RUN ls -l
CMD ./$(echo `ls -l` |awk -F ' ' '{print $NF}')

Jenkinsfile

pipeline {
  agent {
    kubernetes {cloud 'kubernetes'
      cloud 'kubernetes-test'
      slaveConnectTimeout 1200
      workspaceVolume hostPathWorkspaceVolume(hostPath:"/opt/workspace", readOnly: false)
      yaml '''
apiVersion: v1
kind: Pod
spec:
  containers:
    - args: [\'$(JENKINS_SECRET)\', \'$(JENKINS_NAME)\']
      image: 'jenkins/jnlp-slave:latest-jdk11'
      name: jnlp
      imagePullPolicy: IfNotPresent
      volumeMounts:
        - mountPath: "/etc/localtime"
          name: "localtime"
          readOnly: false 
    - command:
        - "cat"
      env:
        - name: "LANGUAGE"
          value: "en_US:en"
        - name: "LC_ALL"
          value: "en_US.UTF-8"
        - name: "LANG"
          value: "en_US.UTF-8"
      image: "ccr.ccs.tencentyun.com/zhubaoe/golang:1.17.13"
      imagePullPolicy: "IfNotPresent"
      name: "build"
      tty: true
      volumeMounts:
        - mountPath: "/etc/localtime"
          name: "localtime"
        - mountPath: "/go/pkg/"
          name: "cachedir"
          readOnly: false
    - command:
        - "cat"
      env:
        - name: "LANGUAGE"
          value: "en_US:en"
        - name: "LC_ALL"
          value: "en_US.UTF-8"
        - name: "LANG"
          value: "en_US.UTF-8"
      image: "registry.cn-beijing.aliyuncs.com/citools/kubectl:self-1.17"
      imagePullPolicy: "IfNotPresent"
      name: "kubectl"
      tty: true
      volumeMounts:
        - mountPath: "/etc/localtime"
          name: "localtime"
          readOnly: false
    - command:
        - "cat"
      env:
        - name: "LANGUAGE"
          value: "en_US:en"
        - name: "LC_ALL"
          value: "en_US.UTF-8"
        - name: "LANG"
          value: "en_US.UTF-8"
      image: "registry.cn-beijing.aliyuncs.com/citools/docker:19.03.9-git"
      imagePullPolicy: "IfNotPresent"
      name: "docker"
      tty: true
      volumeMounts:
        - mountPath: "/etc/localtime"
          name: "localtime"
          readOnly: false
        - mountPath: "/var/run/docker.sock"
          name: "dockersock"
          readOnly: false
  restartPolicy: "Never"
  imagePullSecrets:
  - name: qcloudregistrykey
  nodeSelector:
    build: "true"
  securityContext: {}
  volumes:
    - hostPath:
        path: "/var/run/docker.sock"
      name: "dockersock"
    - hostPath:
        path: "/usr/share/zoneinfo/Asia/Shanghai"
      name: "localtime"
    - name: "cachedir"
      hostPath:
        path: "/opt/gopkg"
    - name: "phpdir"
      hostPath:
        path: "/opt/phppkg"
'''
   }
}
stages {
  stage('Pulling Code') {
	parallel {
	  stage('Pulling Code by Jenkins') {
		when {
		  expression {
			env.giteeBranch == null
		  }
		}
		steps {
		  git(changelog: true, poll: true, url:'https://gitee.com/zhubaoe-go/kobe.git', branch:"${BRANCH}", credentialsId: 'gitee-mima')
   		  script {
   		    COMMIT_ID = sh(returnStdout: true, script: "git log -n 1 --pretty=format:'%h'").trim()
            TMP = sh(returnStdout: true, script: """echo ${SERVICENAME}|awk '{ gsub(/_/,"-"); print \$0 }'""").trim()
            KOBE_NAME = 'mq-' + TMP + '-server'
   		    TAG = BUILD_TAG + '-' + COMMIT_ID
   		    println "Current branch is ${BRANCH}, Commit ID is ${COMMIT_ID}, Image TAG is ${TAG}"

		  }
		}
     }
	  stage('Pulling Code by trigger') {
		when {
		  expression {
			env.giteeBranch != null
		  }
		}
		steps {
		  git(url: 'https://gitee.com/zhubaoe-go/kobe.git', branch: env.giteeBranch, changelog: true, poll: true, credentialsId: 'gitee-mima')
		  script {
			COMMIT_ID = sh(returnStdout: true, script: "git log -n 1 --pretty=format:'%h'").trim()
            TMP = sh(returnStdout: true, script: """echo ${SERVICENAME}|awk '{ gsub(/_/,"-"); print \$0 }'""").trim()
            KOBE_NAME = 'mq-' + TMP + '-server'
			TAG = BUILD_TAG + '-' + COMMIT_ID
			println "Current branch is ${env.giteeBranch}, Commit ID is ${COMMIT_ID}, Image TAG is ${TAG}"
		  }
		}
	}
  }
}
	  stage('Building') {
		steps {
		  container(name: 'build') {
			sh """ 
                go env -w GO111MODULE=on
                go env -w GOPROXY=https://goproxy.io,direct
                go env -w GOPRIVATE=gitee.com/zhubaoe-go/jordan
                export GIT_TERMINAL_PROMPT=1
                go build -o mq_${SERVICENAME}_server mq/${SERVICENAME}_server/main.go
                chmod +x mq_${SERVICENAME}_server
                cp -p mq_${SERVICENAME}_server build-work/k8s2/mq/
			"""
		  }
		}
	  }
	  stage('Docker build for creating image') {
		environment {
		  HARBOR_USER = credentials('registry-secret')
		}
		steps {
		  container(name: 'docker') {
			sh """
              cd build-work/k8s2/mq/
			  echo ${HARBOR_USER_USR} ${HARBOR_USER_PSW} ${TAG}
              docker login -u ${HARBOR_USER_USR} -p ${HARBOR_USER_PSW} ${HARBOR_ADDRESS}
			  docker build -t ${HARBOR_ADDRESS}/${REGISTRY_DIR}/${IMAGE_NAME}:${TAG} --build-arg SERVICENAME=${SERVICENAME} .
			  docker push ${HARBOR_ADDRESS}/${REGISTRY_DIR}/${IMAGE_NAME}:${TAG}
			"""
		  }
		}
	  }
	  stage('Deploying to K8s') {
		environment {
		  MY_KUBECONFIG = credentials('k8s-config')
		}
		steps {
		  container(name: 'kubectl'){
			sh """
			  /usr/local/bin/kubectl --kubeconfig $MY_KUBECONFIG set image deploy -l app="${KOBE_NAME}" "${KOBE_NAME}"=${HARBOR_ADDRESS}/${REGISTRY_DIR}/${IMAGE_NAME}:${TAG} -n $NAMESPACE
			"""
		  }
		}
	  }
	}
  environment {
    COMMIT_ID = ""
    HARBOR_ADDRESS = "ccr.ccs.tencentyun.com"
    REGISTRY_DIR = "zhubaoe"
    IMAGE_NAME = "kobe"
    NAMESPACE = "${ENV}"
    TAG = ""
    KOBE_NAME = ""
}
  parameters {
	gitParameter(branch: '', branchFilter: 'origin/(.*)', defaultValue: 'feature/test', description: 'Branch for build and deploy', name:'BRANCH', quickFilterEnabled: false, selectedValue: 'NONE', sortMode: 'NONE', tagFilter: '*', type: 'PT_BRANCH')
  }
}


特殊处理主要是这个地方

TMP = sh(returnStdout: true, script: """echo ${SERVICENAME}|awk '{ gsub(/_/,"-"); print \$0 }'""").trim()

使用awk的gsub函数,将服务名里面的所有下划线改为中划线,最开始是用的sub函数,而sub函数默认只会匹配修改遇到的第一个下划线,所以使用gsub,全局匹配,做个记录,免得忘了!!

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
在 Web 开发的世界里,敏捷才是王道。使用更少的费用和资源来更快地完成网站和网络应用,从而获得更多的竞争优势。此外,他们不仅希望快速完成 Web 开发,对可用性和用户体验的要求也很高。这需要开发更多的功能和高级编程语言来开发网站功能,例如 Golang。如果您正在寻找用于 Web 编程、移动开发、微服务和 ERP 系统的强大工具,我们相信,您应该切换到 Golang 的原因是:1. 简单性如果 Python 是迄今为止容易学习的编程语言,那么 Golang 甚至更简单。学习 Go 语言要比学习 Python 快得多。一些开发人员声称 Golang 几乎和 JavaScript 一样简单。 2. 高级编译能力Golang 是一个编译执行语言,它能够直接将代码编译成可执行的二进制机器码。Golang 静态地将所有依赖库和模块编译到到一个二进制文件中。你不再需要在服务器上安装任何依赖项——你所需要做的就是上传一个编译的文件,你的应用就可以简单的运行了。 3. 并发和性能Golang 并发模型能够确保更高的性能(性能甚至 2x、3x 的提升)。大多数现代编程语言都支持并发,但是 Golang 采用了更节省资源的方法。Golang 提供更高性能的goroutine模型来支持多线程,goroutine能够更加节省 CPU 和内存资源。所以 Golang 更有助于降低成本和资源 4. 框架和库的本地支持Golang sdk 提供了功能丰富的 api,因此不用过多的依赖第三方库就能很好的使用 Golang。当然,如果你需要的话,你可以下载很多工具和框架(Golang 周围的社区已经很强大很完善了),但是常用的 API 都已经内置到 Golang 核心库中了。这样可以加快整个 Golang Web 开发过程的速度,并且使您无需寻找任何第三方依赖。 5.  IDE 和调试Golang 创作者在创建具有先进调试工具和插件的先进的集成开发环境方面做得很好。这可以说是编程中最关键的方面,因为 IDE 会严重阻碍或加速开发过程。今天,当敏捷性给软件公司带来竞争优势时,伟大的 IDE 和调试工具是一个非常重要的优势。 6. 清晰的语法、它包含零不必要的组件。Go是建立在实用的头脑中的:而不是必须深入研究语言结构,开发者现在可以自由地专注于开发本身。本质上Golang 是非常务实的。 本课程基于电商业务,采用前后端分离方式进行构建和讲解,后端主要包括:Gin+Go-Micro以及前端主要包括Vue。由于内容比较多,课程会分模块讲解,包括用户服务模块、商品服务模块、订单服务模块、消息服务模块、支付服务模块、后台管理系统模块、营销中心模块、物流系统模块、搜索中心模块、智能推荐模块等等。本课程内容为用户服务模块,全套课程将多种经过工业界验证的成熟技术解决方案呈现给大家,本套课程拒绝枯燥的理论,全程代码实操,通过项目驱动的方式,让大家能够真实体验Go语言在工业界开发过程,帮助大家建立Go语言开发思维,学习本套课程全部内容你完全可以自主开发一套高性能高可用高扩展的Go电商系统。本套案例集后端+前台+测试+运维一体,多方位的带你熟悉Go语言开发的全过程。本课程包含的技术:开发工具为:GOLAND、WebStormGin+Go-Micro Vue+Nodejs+jQuery+ElmentUIRedisRabbitMQ等等

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值