kubernetes部署maven
文章目录
提供者:MappleZF
版本:1.0.0
一、下载解压软件
maven官方地址:https://maven.apache.org/
阿里云镜像站点: https://maven.aliyun.com/
下载链接:https://archive.apache.org/dist/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.tar.gz
[root@k8smaster01:/data/tools]# wget -c https://archive.apache.org/dist/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.tar.gz
解压至jenkins安装目录
(注意jenkins镜像中的JDK版本)
mkdir -p /mnt/cephfs/cephfs-pvjenkins/maven-3.6.3-8u252
tar -xzf apache-maven-3.6.3-bin.tar.gz -C /mnt/cephfs/cephfs-pvjenkins/maven-3.6.3-8u252
cd /mnt/cephfs/cephfs-pvjenkins/maven-3.6.3-8u252
mv apache-maven-3.6.3/* ./
rm -rf apache-maven-3.6.3
设置国内镜像源
在/mnt/cephfs/cephfs-pvjenkins/maven-3.6.3-8u252/conf/settings.xml
在<mirrors></mirrors>标签中添加 mirror 子节点: 约在155行
[root@k8smaster01:/mnt/cephfs/cephfs-pvjenkins/maven-3.6.3-8u252/conf]# vim settings.xml
<mirror>
<id>aliyunmaven</id>
<mirrorOf>*</mirrorOf>
<name>aliyun repository</name>
<url>https://maven.aliyun.com/repository/public</url>
</mirror>
参考:
<mirrors>
<mirror>
<id>aliyunmaven</id>
<mirrorOf>*</mirrorOf>
<name>aliyun repository</name>
<url>https://maven.aliyun.com/repository/public</url>
</mirror>
<!-- mirror
| Specifies a repository mirror site to use instead of a given repository. The repository that
| this mirror serves has an ID that matches the mirrorOf element of this mirror. IDs are used
| for inheritance and direct lookup purposes, and must be unique across the set of mirrors.
|
<mirror>
<id>mirrorId</id>
<mirrorOf>repositoryId</mirrorOf>
<name>Human Readable Name for this Mirror.</name>
<url>http://my.repository.com/repo/path</url>
</mirror>
-->
</mirrors>
二、制作微服务的底包镜像
2.1 自定义Dockerfile
[root@k8smaster01:/data/Dockerfile/dubbo]# vim Dockerfile
FROM harbor.iot.com/public/jre8:8u112
RUN /bin/cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && \
echo 'Asia/Shanghai' > /etc/timezone
ADD config.yml /opt/prometheus/config.yml
ADD jmx_prometheus_javaagent-0.13.0.jar /opt/prometheus/
WORKDIR /opt/project_dir
ADD entrypoint.sh /entrypoint.sh
CMD [ "/entrypoint.sh" ]
2.2 编写附件
编写 #config.yml
[root@k8smaster01:/data/Dockerfile/dubbo]# vim config.yml
---
rules:
- pattern: '.*'
下载jmx_javaagent-0.3.1jar
[root@k8smaster01:/data/Dockerfile/dubbo]# wget -c https://repo1.maven.org/maven2/io/prometheus/jmx/jmx_prometheus_javaagent/0.13.0/jmx_prometheus_javaagent-0.13.0.jar
编写 #entrypoint.sh
[root@k8smaster01:/data/Dockerfile/dubbo]# vim entrypoint.sh
#!/bin/sh
M_OPTS="-Duser.timezone=Asia/Shanghai -javaagent:/opt/prometheus/jmx_prometheus_javaagent-0.13.0.jar=$(hostname -i):${M_PORT:-"12346"}:/opt/prometheus/config.yml"
C_OPTS=${C_OPTS}
JAR_BALL=${JAR_BALL}
exec java -jar ${M_OPTS} ${C_OPTS} ${JAR_BALL}
[root@k8smaster01:/data/Dockerfile/dubbo]# chmod +x entrypoint.sh
2.3 构建镜像
[root@k8smaster01:/data/Dockerfile/dubbo]# docker build -t harbor.iot.com/base/jre8:8u112 ./
[root@k8smaster01:/data/Dockerfile/dubbo]# docker push harbor.iot.com/base/jre8:8u112
三、jinkens界面操作
3.1 参数化构建过程
字符参数:
名称: app_name
默认值:
描述: 项目的名称, 例:dubbo-demo-service
字符参数:
名称: image_name
默认值:
描述: docker镜像的名称,例:app/dubbo-demo-service
字符参数:
名称: git_repo
默认值:
描述: 项目所在的git中央仓库的地址,例:https://go.lowaniot.com/root/dubbo-demo-service.git
字符参数:
名称: git_ver
默认值:
描述: 项目在git中央仓库所对应的,项目分支或者版本号
字符参数:
名称: add_tag
默认值:
描述: docker镜像标签的一部分,日期时间戳,例如:20200817_2119
字符参数:
名称: mvn_dir
默认值: ./
描述: 编译项目的目录,默认为项目的根目录
字符参数:
名称: target_dir
默认值: ./target
描述: 编译完成项目后,产生的jar/war包所在的目录
字符参数:
名称: mvn_cmd
默认值: mvn clean package -Dmaven.test.skip=true
描述: 执行编译所用的命令
字符参数:
名称: base_image
默认值: base/jre:8u112
描述: 项目使用的docker底包镜像
字符参数:
名称: maven
默认值: 3.6.3-8u252
描述: 执行编译使用的maven软件版本
3.2 流水线自定义脚本
Pipeline script脚本如下:
pipeline {
agent any
stages {
stage('pull') { //get project code from repo
steps {
sh "git clone ${params.git_repo} ${params.app_name}/${env.BUILD_NUMBER} && cd ${params.app_name}/${env.BUILD_NUMBER} && git checkout ${params.git_ver}"
}
}
stage('build') { //exec mvn cmd
steps {
sh "cd ${params.app_name}/${env.BUILD_NUMBER} && /var/jenkins_home/${params.maven}/bin/${params.mvn_cmd}"
}
}
stage('package') { //move jar file into project_dir
steps {
sh "cd ${params.app_name}/${env.BUILD_NUMBER} && cd ${params.target_dir} && mkdir project_dir && mv *.jar ./project_dir"
}
}
stage('image') { //build image and push to registry
steps {
writeFile file:"${params.app_name}/${env.BUILD_NUMBER}/Dockerfile", text: """FROM harbor.iot.com/${params.base_image}
ADD ${parms.target_dir}/project_dir /opt/project_dir"""
sh "cd ${params.app_name}/${env.BUILD_NUMBER} && docker build -t harbor.iot.com/${params.image_name}:${params.git_ver}_${params.add_tag} . && docker push harbor.iot.com/${params.image_name}:${params.git_ver}_${params.add_tag}"
}
}
}
}
四、示例交付服务至kubernetes
4.1 交付dubbo-demo-service到kubernetes
创建资源配置清单
#创建namespace
[root@k8smaster01:/data/yaml/dubbo-demo-service]# vim namespace.yaml
apiVersion: v1
kind: Namespace
metadata:
name: app
spec: {}
status: {}
[root@k8smaster01:/data/yaml/dubbo-demo-service]# kubectl apply -f namespace.yaml
#创建secert
[root@k8smaster01:/data/yaml/dubbo-demo-service]# kubectl create secret docker-registry harbor --docker-server=harbor.iot.com --docker-username=admin --docker-password=appleMysql -n app
或者
[root@k8smaster01:/data/yaml/dubbo-demo-service]# vim secret.yaml
apiVersion: v1
data:
.dockerconfigjson: eyJhdXRocyI6eyJoYXJib3IuaW90LmNvbSI6eyJ1c2VybmFtZSI6ImFkbWluIiwicGFzc3dvcmQiOiJhcHBsZU15c3FsIiwiYXV0aCI6IllXUnRhVzQ2WVhCd2JHVk5lWE54YkE9PSJ9fX0=
kind: Secret
metadata:
creationTimestamp: null
name: harbor
namespace: app
type: kubernetes.io/dockerconfigjson
[root@k8smaster01:/data/yaml/dubbo-demo-service]# kubectl apply -f secret.yaml
#创建deployment
[root@k8smaster01:/data/yaml/dubbo-demo-service]# vim deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: dubbo-demo-service
namespace: app
labels:
name: dubbo-demo-service
spec:
replicas: 1
selector:
matchLabels:
name: dubbo-demo-service
template:
metadata:
labels:
app: dubbo-demo-service
name: dubbo-demo-service
spec:
containers:
- name: dubbo-demo-service
image: harbor.iot.com/app/dubbo-demo-service:master_0818
ports:
- containerPort: 20880
protocol: TCP
env:
- name: JAR_BALL
value: dubbo-server.jar
imagePullPolicy: IfNotPresent
imagePullSecrets:
- name: harbor
restartPolicy: Always
terminationGracePeriodSeconds: 30
securityContext:
runAsUser: 0
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
revisionHistoryLimit: 7
progressDeadlineSeconds: 600
~
4.2 部署dubbo-monitor
4.2.1 用于监控zookeeper
GitHub地址:https://github.com/Jeromefromcn/dubbo-monitor
[root@k8smaster01:/root]# unzip dubbo-monitor-master.zip
[root@k8smaster01:/root]# cd /root/dubbo-monitor-master/dubbo-monitor-simple/conf
[root@k8smaster01:/root/dubbo-monitor-master/dubbo-monitor-simple/conf]# vim dubbo_origin.properties
dubbo.container=log4j,spring,registry,jetty
dubbo.application.name=dubbo-monitor
dubbo.application.owner=iot
dubbo.registry.address=zookeeper://10.106.167.199:2181
#dubbo.registry.address=zookeeper://127.0.0.1:2181
#dubbo.registry.address=redis://127.0.0.1:6379
#dubbo.registry.address=dubbo://127.0.0.1:9090
dubbo.protocol.port=20880
dubbo.jetty.port=8080
dubbo.jetty.directory=/dubbo-monitor-simple/monitor
dubbo.charts.directory=/dubbo-monitor-simple/charts
dubbo.statistics.directory=/dubbo-monitor-simple/statistics
dubbo.log4j.file=logs/dubbo-monitor-simple.log
dubbo.log4j.level=WARN
~
注:更改bin/start.sh
执行:
[root@k8smaster01:/root/dubbo-monitor-master]# sed -r -i -e '/^nohup/{p;:a;N;$!ba;d}' ./dubbo-monitor-simple/bin/start.sh && sed -r -i -e "s%^nohup(.*)%exec \1%" ./dubbo-monitor-simple/bin/start.sh
[root@k8smaster01:/root]# cp -a dubbo-monitor-master /data/Dockerfile/
[root@k8smaster01:/data/Dockerfile]# cd dubbo-monitor-master/
4.2.2 构建镜像
[root@k8smaster01:/data/Dockerfile/dubbo-monitor-master]# docker build -t harbor.iot.com/public/dubbo-monitor:v1.0.0 ./
[root@k8smaster01:/data/Dockerfile/dubbo-monitor-master]# docker push harbor.iot.com/public/dubbo-monitor:v1.0.0
4.2.3 制作交付k8s的资源配置清单
制作configmap
[root@k8smaster01:/data/yaml/dubbo-monitor-master]# vim configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: dubbo-monitor-properties
namespace: devops
data:
dubbo.properties: |
dubbo.container=log4j,spring,registry,jetty
dubbo.application.name=dubbo-monitor
dubbo.application.owner=iot
dubbo.registry.address=zookeeper://10.106.167.199:2181
dubbo.protocol.port=20880
dubbo.jetty.port=8080
dubbo.jetty.directory=/dubbo-monitor-simple/monitor
dubbo.charts.directory=/dubbo-monitor-simple/charts
dubbo.statistics.directory=/dubbo-monitor-simple/statistics
dubbo.log4j.file=/dubbo-monitor-simple/logs/dubbo-monitor-simple.log
dubbo.log4j.level=WARN
~
[root@k8smaster01:/data/yaml/dubbo-monitor-master]# kubectl apply -f configmap.yaml
制作svc
[root@k8smaster01:/data/yaml/dubbo-monitor-master]# vim svc.yaml
apiVersion: v1
kind: Service
metadata:
name: dubbo-monitor
namespace: devops
spec:
ports:
- protocol: TCP
port: 8080 #此处的port需要和Ingress的servicePort对应
targetPort: 8080
selector:
app: dubbo-monitor
~
[root@k8smaster01:/data/yaml/dubbo-monitor-master]# kubectl apply -f svc.yaml
制作ingress
[root@k8smaster01:/data/yaml/dubbo-monitor-master]# vim ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: dubbo-monitor
namespace: devops
spec:
rules:
- host: dubbo-monitor.iot.com
http:
paths:
- path: /
backend:
serviceName: dubbo-monitor
servicePort: 8080
[root@k8smaster01:/data/yaml/dubbo-monitor-master]# kubectl apply -f ingress.yaml
制作deployment
[root@k8smaster01:/data/yaml/dubbo-monitor-master]# vim deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: dubbo-monitor
namespace: devops
labels:
name: dubbo-monitor
spec:
replicas: 1
selector:
matchLabels:
name: dubbo-monitor
template:
metadata:
labels:
app: dubbo-monitor
name: dubbo-monitor
spec:
containers:
- name: dubbo-monitor
image: harbor.iot.com/public/dubbo-monitor:v1.0.0
ports:
- containerPort: 8080
protocol: TCP
- containerPort: 20880
protocol: TCP
imagePullPolicy: IfNotPresent
volumeMounts:
- name: configmap-volume-dubbo-monitor
mountPath: /dubbo-monitor-simple/conf
volumes:
- name: configmap-volume-dubbo-monitor
configMap:
name: dubbo-monitor-properties
imagePullSecrets:
- name: harbor
restartPolicy: Always
terminationGracePeriodSeconds: 30
securityContext:
runAsUser: 0
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
revisionHistoryLimit: 7
progressDeadlineSeconds: 300
~
[root@k8smaster01:/data/yaml/dubbo-monitor-master]# kubectl apply -f deployment.yaml
4.3 解析DNS
# vim lowan.com.zone
添加
dubbo-monitor A 192.168.13.100
:x 保存退出
systemctl restart named
dig -t A dubbo-monitor.lowan.com @192.168.13.100 + short
浏览器访问:http://dubbo-monitor.lowan.com/
后续参考(集群):
01 kubernetes二进制部署
02 kubernetes辅助环境设置
03 K8S集群网络ACL规则
04 Ceph集群部署
05 部署zookeeper和kafka集群
06 部署日志系统
07 部署Indluxdb-telegraf
08 部署jenkins
09 部署k3s和Helm-Rancher
10 部署maven软件