K8s + SpringBoot实现零宕机发布:健康检查+滚动更新+优雅停机+弹性伸缩
配置
健康检查
- 健康检查类型:就绪探针(readiness)+ 存活探针(liveness)
- 探针类型:exec(进入容器执行脚本)、tcpSocket(探测端口)、httpGet(调用接口)
这里使用httpGet
代码配置
pom.xml依赖
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
配置文件定义访问端口、路径及权限 application.yaml
spring:
application:
name: k8s-demo
server:
port: 8080
management:
server:
port: 50000 # 使用独立的运维端口,此端口不对外暴露
endpoint:
health:
probes:
enabled: true
endpoints:
web:
base-path: /actuator
exposure:
include: health
此时,将暴露/actuator/health/readiness和/actuator/health/liveness两个接口,访问方式如下:
curl -X GET http://127.0.0.1:50000/actuator/health/readiness
curl -X GET http://127.0.0.1:50000/actuator/health/liveness
k8s部署模版
k8s-demo.yaml
apiVersion: apps/v1
kind: Deployment
spec:
template:
spec:
containers:
- name: k8s-demo # app名称
image: 192.168.68.133:5000/k8s-demo:latest # 此为镜像私服地址
imagePullPolicy: Always
ports:
- containerPort: 8080 # 应用端口
- name: management-port
containerPort: 50000 # 应用管理端口
readinessProbe: # 就绪探针
httpGet:
path: /actuator/health/readiness
port: management-port
initialDelaySeconds: 30 # 延迟加载时间
periodSeconds: 10 # 重试时间间隔
timeoutSeconds: 1 # 超时时间设置
successThreshold: 1 # 健康阈值
failureThreshold: 6 # 不健康阈值
livenessProbe: # 存活探针
httpGet:
path: /actuator/health/liveness
port: management-port
initialDelaySeconds: 30 # 延迟加载时间
periodSeconds: 10 # 重试时间间隔
timeoutSeconds: 1 # 超时时间设置
successThreshold: 1 # 健康阈值
failureThreshold: 6 # 不健康阈值
优雅停机
application.yaml配置
spring:
lifecycle:
timeout-per-shutdown-phase: 30s
server:
shutdown: graceful
management:
endpoint:
shutdown:
enabled: true
endpoints:
web:
exposure:
include: shutdown
此时将暴露以下接口:
curl -X POST http://127.0.0.1:50000/actuator/shutdown
确保项目Dockerfile模板集成curl工具,否则无法使用curl命令
FROM openjdk:17-jdk-alpine
#构建参数
ARG JAR_FILE="k8s-demo.jar"
ARG WORK_PATH="/app"
ARG EXPOSE_PORT=8080
#环境变量
ENV JAVA_OPTS=""\
JAR_FILE=${JAR_FILE}
#设置时区
RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && echo 'Asia/Shanghai' > /etc/timezone
RUN sed -i 's/dl-cdn.alpinelinux.org/mirrors.ustc.edu.cn/g' /etc/apk/repositories \
&& apk add --no-cache curl
#将maven目录的jar包拷贝到docker中
COPY target/$JAR_FILE $WORK_PATH/
#设置工作目录
WORKDIR $WORK_PATH
# 指定于外界交互的端口
EXPOSE $EXPOSE_PORT
# 配置容器,使其可执行化
ENTRYPOINT exec java $JAVA_OPTS -jar $JAR_FILE
如果是使用的docker-maven-plugin插件配置,则可以如下:
<properties>
<docker.image.prefix>192.168.68.133:5000</docker.image.prefix>
<docker.registry>192.168.68.133</docker.registry>
</properties>
<build>
<finalName>${project.artifactId}</finalName>
<plugins>
<!-- docker 插件 begin -->
<plugin>
<groupId>com.spotify</groupId>
<artifactId>docker-maven-plugin</artifactId>
<version>1.2.2</version>
<configuration>
<!--指定生成的镜像名-->
<imageName>${docker.image.prefix}/${project.artifactId}:${project.version}</imageName>
<!--指定基础镜像jdk17-->
<baseImage>openjdk:17-jdk-alpine</baseImage>
<!--切换到app目录-->
<workdir>/app</workdir>
<env>
<TZ>Asia/Shanghai</TZ>
</env>
<runs>
<!--时区配置-->
<run>ln -sf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone</run>
<!--集成curl-->
<run>sed -i 's/dl-cdn.alpinelinux.org/mirrors.ustc.edu.cn/g' /etc/apk/repositories && apk add --no-cache curl</run>
</runs>
<!--暴露端口-->
<exposes>8080</exposes>
<entryPoint>["java","-jar","/${project.build.finalName}.jar"]</entryPoint>
<!--仓库地址-->
<registryUrl>http://${docker.registry}:5000</registryUrl>
<!--指定远程 docker api地址-->
<dockerHost>http://${docker.registry}:2375</dockerHost>
<!--是否推送镜像-->
<pushImage>true</pushImage>
<!--推送后是否覆盖已存在的标签镜像-->
<forceTags>true</forceTags>
<resources>
<resource>
<targetPath>/</targetPath>
<!-- 目录地址,也就是target生成的地址 -->
<directory>${project.build.directory}</directory>
<!--需要复制的jar包 -->
<include>${project.build.finalName}.jar</include>
</resource>
</resources>
</configuration>
</plugin>
</plugins>
</build>
也可以 docker-maven-plugin 和 Dockerfile 结合使用
此时,k8s部署模板可以添加停机回调
apiVersion: apps/v1
kind: Deployment
spec:
template:
spec:
containers:
- name: k8s-demo
image: 192.168.68.133:5000/k8s-demo:latest
imagePullPolicy: Always
ports:
- containerPort: 8080
- containerPort: 50000
lifecycle:
preStop: # 结束回调钩子
exec:
command: ["curl", "-XPOST", "127.0.0.1:50000/actuator/shutdown"]
java项目也可以省略结束回调钩子的配置
滚动更新
k8s资源调度滚动更新策略配置,若要实现零宕机发布,需支持优雅停机
apiVersion: apps/v1
kind: Deployment
metadata:
name: k8s-demo
labels:
app: k8s-demo
spec:
selector:
matchLabels:
app: k8s-demo
replicas: 3 # Pod副本数
strategy:
type: RollingUpdate # 滚动更新策略
rollingUpdate:
maxSurge: 1 # 升级过程中最多可以比原先设置的副本数多出的数量
maxUnavailable: 0 # 升级过程中最多有多少个POD处于无法提供服务的状态
弹性伸缩
为pod设置资源限制后,创建HPA
apiVersion: apps/v1
kind: Deployment
metadata:
name: k8s-demo
labels:
app: k8s-demo
spec:
template:
spec:
containers:
- name: k8s-demo
image: 192.168.68.133:5000/k8s-demo:latest
imagePullPolicy: Always
resources: # 容器资源管理
limits: # 资源限制(监控使用情况)
cpu: 0.5
memory: 1Gi
requests: # 最小可用资源(灵活调度)
cpu: 0.15
memory: 300Mi
---
kind: HorizontalPodAutoscaler # 弹性伸缩控制器
apiVersion: autoscaling/v2
metadata:
name: k8s-demo
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: k8s-demo
minReplicas: 3 # 缩放范围
maxReplicas: 6
metrics:
- type: Resource
resource:
name: cpu # 指定资源指标
target:
type: Utilization
averageUtilization: 50
汇总配置
代码application.yml配置
spring:
application:
name: k8s-demo
lifecycle:
timeout-per-shutdown-phase: 30s
server:
port: 8080
shutdown: graceful
management:
server:
port: 50000 # 使用独立的运维端口,此端口不对外暴露
endpoint:
shutdown:
enabled: true
health:
probes:
enabled: true
endpoints:
web:
base-path: /actuator
exposure:
include: health,shutdown
k8s部署模板k8s-demo.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: k8s-demo
labels:
app: k8s-demo
spec:
selector:
matchLabels:
app: k8s-demo
replicas: 3 # Pod副本数
strategy:
type: RollingUpdate # 滚动更新策略
rollingUpdate:
maxSurge: 1 # 升级过程中最多可以比原先设置的副本数多出的数量
maxUnavailable: 0 # 升级过程中最多有多少个POD处于无法提供服务的状态
template:
metadata:
name: k8s-demo
labels:
app: k8s-demo
spec:
affinity: # 设置调度策略,采取多主机/多可用区部署
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app
operator: In
values:
- k8s-demo
topologyKey: "kubernetes.io/hostname" # 多可用区为"topology.kubernetes.io/zone"
terminationGracePeriodSeconds: 30 # 优雅终止宽限期
containers:
- name: k8s-demo
image: 192.168.68.133:5000/k8s-demo:latest # 此为镜像私服地址
imagePullPolicy: Always
ports:
- containerPort: 8080 # 应用端口
- name: management-port
containerPort: 50000 # 应用管理端口
readinessProbe: # 就绪探针
httpGet:
path: /actuator/health/readiness
port: management-port
initialDelaySeconds: 30 # 延迟加载时间
periodSeconds: 10 # 重试时间间隔
timeoutSeconds: 1 # 超时时间设置
successThreshold: 1 # 健康阈值
failureThreshold: 9 # 不健康阈值
livenessProbe: # 存活探针
httpGet:
path: /actuator/health/liveness
port: management-port
initialDelaySeconds: 30 # 延迟加载时间
periodSeconds: 10 # 重试时间间隔
timeoutSeconds: 1 # 超时时间设置
successThreshold: 1 # 健康阈值
failureThreshold: 6 # 不健康阈值
resources: # 容器资源管理
limits: # 资源限制(监控使用情况)
cpu: 0.5
memory: 1Gi
requests: # 最小可用资源(灵活调度)
cpu: 0.15
memory: 300Mi
env:
- name: TZ
value: Asia/Shanghai
---
kind: HorizontalPodAutoscaler # 弹性伸缩控制器
apiVersion: autoscaling/v2
metadata:
name: k8s-demo
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: k8s-demo
minReplicas: 3 # 缩放范围
maxReplicas: 6
metrics:
- type: Resource
resource:
name: cpu # 指定资源指标
target:
type: Utilization
averageUtilization: 50