云原生—构建—发布—部署—Spring Boot Dubbo 微服务—K8S
前置说明
上篇文章,通过 Kind和Helm 搭建了本地开发环境,部署了 Redis/Zookeeper/MySQL 服务,
本篇把本地开发的 Spring Boot Dubbo 微服务应用,一键构建打标发布镜像,并部署到 K8S.
一键构建打标发布镜像
拉镜像
runImage and builder
paketobuildpacks/run:base-cnb
paketobuildpacks/builder:base
pullPolicy = "IF_NOT_PRESENT"
runImage = "paketobuildpacks/run:base-cnb"
builder = "paketobuildpacks/builder:base"
打标镜像
imageName
imageName = "registry.cn-shenzhen.aliyuncs.com/uid13/${project.name}:${project.version}"
发布镜像到 Aliyun
docker {
publishRegistry {
url = "registry.cn-shenzhen.aliyuncs.com"
username = "你的账号"
password = "你的密码"
}
}
publish = true
备注
若不想发布,直接在 K8S 用,则执行
kind load docker-image demo-app:1.0-SNAPSHOT --name default
Dependency Mappings
构建镜像,默认会下载:
https://github.com/bell-sw/Liberica/releases/download/17.0.1+12/bellsoft-jre17.0.1+12-linux-amd64.tar.gz
https://github.com/anchore/syft/releases/download/v0.33.0/syft_0.33.0_linux_amd64.tar.gz
https://repo.spring.io/release/org/springframework/cloud/spring-cloud-bindings/1.8.0/spring-cloud-bindings-1.8.0.jar
在国内,这些文件是需要科学上网才能成功下载的,而 CNB(Cloud Native Builder) 提供了 Dependency Mappings 改变下载地址.
file 协议,从本地下载
创建自定义目录结构
PS D:\IdeaProjects\reproduce-demo> tree /f cnb
卷 Data 的文件夹 PATH 列表
卷序列号为 000000BD EE47:E3A1
D:\IDEAPROJECTS\REPRODUCE-DEMO\CNB
└─bindings
├─binary
│ bellsoft-jre17.0.1+12-linux-amd64.tar.gz
│ spring-cloud-bindings-1.8.0.jar
│ syft_0.33.0_linux_amd64.tar.gz
│
└─mapping
3ac79e13f80582bb2289ded782d1d43aafcb7041be08cc9f2e8796b4f3f6262d
9b937fe5dfe37547cef0b5952f38ef098d8cb6307763e4c4036eff315614bb93
c75ef3152817c28c7049d7e0052dfba3ed461a7151984e594b656c0be927888e
type
科学上网下载放入 cnb/bindings/binary,mapping 下创建 type 文件,内容为 dependency-mapping
获取 sha256Hex
import cn.hutool.core.io.FileUtil;
import cn.hutool.crypto.digest.DigestUtil;
import lombok.extern.slf4j.Slf4j;
import java.io.File;
import java.util.List;
@Slf4j
public class PrintData {
public static void main(String[] args) {
String path = "D:\\IdeaProjects\\reproduce-demo\\cnb\\bindings\\binary";
List<File> files = FileUtil.loopFiles(path);
files.forEach(file -> {
String sha256Hex = DigestUtil.sha256Hex(file);
log.info("File: {}, sha256Hex: {} \n", file.getName(), sha256Hex);
});
}
}
14:14:11.031 [main] INFO com.lab.util.PrintData - File: bellsoft-jre17.0.1+12-linux-amd64.tar.gz, sha256Hex: 9b937fe5dfe37547cef0b5952f38ef098d8cb6307763e4c4036eff315614bb93
14:14:11.041 [main] INFO com.lab.util.PrintData - File: spring-cloud-bindings-1.8.0.jar, sha256Hex: c75ef3152817c28c7049d7e0052dfba3ed461a7151984e594b656c0be927888e
14:14:11.197 [main] INFO com.lab.util.PrintData - File: syft_0.33.0_linux_amd64.tar.gz, sha256Hex: 3ac79e13f80582bb2289ded782d1d43aafcb7041be08cc9f2e8796b4f3f6262d
以 sha256Hex 创建文件,并写入下载地址,
例如:bellsoft-jre17.0.1+12-linux-amd64.tar.gz
文件 9b937fe5dfe37547cef0b5952f38ef098d8cb6307763e4c4036eff315614bb93
内容为 file:///platform/bindings/binary/bellsoft-jre17.0.1+12-linux-amd64.tar.gz
设置绑定
def myBinding = file("${project.rootDir}/cnb/bindings").getPath() + ":" + "/platform/bindings"
binding(myBinding)
JVM 相关设置
def prepend = new StringJoiner(" ")
// e.g. 通过 k8s nfs/nas 挂载 skywalking-agent
// Helm 安装 SkyWalking https://archive.apache.org/dist/skywalking/kubernetes/4.4.0/
// prepend.add("-javaagent:/path/to/skywalking-agent.jar")
// prepend.add("-Dskywalking.agent.service_name=${project.name}")
// prepend.add("-Dskywalking.collector.backend_service=oap:11800")
def append = "-Xms128m -Xmx128m"
// @see https://paketo.io/docs/howto/configuration/
environment = [
'BP_OCI_AUTHORS' : "Jazz",
'BP_JVM_VERSION' : '17',
'BPE_DELIM_JAVA_TOOL_OPTIONS' : ' ',
'BPE_PREPEND_JAVA_TOOL_OPTIONS': prepend.toString(),
'BPE_APPEND_JAVA_TOOL_OPTIONS' : append
]
整体 gradle bootBuildImage
// @see https://docs.spring.io/spring-boot/docs/2.6.13/gradle-plugin/reference/htmlsingle/
tasks.bootBuildImage {
buildCache {
volume {
name = "cache-${project.name}.build"
}
}
launchCache {
volume {
name = "cache-${project.name}.launch"
}
}
imageName = "registry.cn-shenzhen.aliyuncs.com/uid13/${project.name}:${project.version}"
docker {
publishRegistry {
url = "registry.cn-shenzhen.aliyuncs.com"
username = "你的账号"
password = "你的密码"
}
}
publish = true
pullPolicy = "IF_NOT_PRESENT"
runImage = "paketobuildpacks/run:base-cnb"
builder = "paketobuildpacks/builder:base"
// 关于: Dependency Mappings
// @see https://github.com/paketo-buildpacks/paketo-website/blob/main/content/docs/howto/configuration.md
def myBinding = file("${project.rootDir}/cnb/bindings").getPath() + ":" + "/platform/bindings"
binding(myBinding)
def prepend = new StringJoiner(" ")
// e.g. 通过 k8s nfs/nas 挂载 skywalking-agent
// Helm 安装 SkyWalking https://archive.apache.org/dist/skywalking/kubernetes/4.4.0/
// prepend.add("-javaagent:/path/to/skywalking-agent.jar")
// prepend.add("-Dskywalking.agent.service_name=${project.name}")
// prepend.add("-Dskywalking.collector.backend_service=oap:11800")
def append = "-Xms128m -Xmx128m"
// @see https://paketo.io/docs/howto/configuration/
environment = [
'BP_OCI_AUTHORS' : "Jazz",
'BP_JVM_VERSION' : '17',
'BPE_DELIM_JAVA_TOOL_OPTIONS' : ' ',
'BPE_PREPEND_JAVA_TOOL_OPTIONS': prepend.toString(),
'BPE_APPEND_JAVA_TOOL_OPTIONS' : append
]
}
执行任务
> Task :demo-app:bootBuildImage
Building image 'registry.cn-shenzhen.aliyuncs.com/uid13/demo-app:1.0-SNAPSHOT'
> Executing lifecycle version v0.13.2
> Using build cache volume 'cache-demo-app.build'
> Running creator
[creator] ===> DETECTING
[creator] 8 of 19 buildpacks participating
[creator] paketo-buildpacks/ca-certificates 3.0.2
[creator] paketo-buildpacks/bellsoft-liberica 9.0.2
[creator] paketo-buildpacks/syft 1.3.1
[creator] paketo-buildpacks/executable-jar 6.0.2
[creator] paketo-buildpacks/dist-zip 5.0.2
[creator] paketo-buildpacks/spring-boot 5.3.0
[creator] paketo-buildpacks/environment-variables 4.0.1
[creator] paketo-buildpacks/image-labels 4.0.2
[creator] ===> ANALYZING
[creator] Previous image with name "registry.cn-shenzhen.aliyuncs.com/uid13/demo-app:1.0-SNAPSHOT" not found
[creator] ===> RESTORING
[creator] ===> BUILDING
[creator]
[creator] Paketo CA Certificates Buildpack 3.0.2
[creator] https://github.com/paketo-buildpacks/ca-certificates
[creator] Launch Helper: Contributing to layer
[creator] Creating /layers/paketo-buildpacks_ca-certificates/helper/exec.d/ca-certificates-helper
[creator]
[creator] Paketo BellSoft Liberica Buildpack 9.0.2
[creator] https://github.com/paketo-buildpacks/bellsoft-liberica
[creator] Build Configuration:
[creator] $BP_JVM_TYPE JRE the JVM type - JDK or JRE
[creator] $BP_JVM_VERSION 17 the Java version
[creator] Launch Configuration:
[creator] $BPL_DEBUG_ENABLED false enables Java remote debugging support
[creator] $BPL_DEBUG_PORT 8000 configure the remote debugging port
[creator] $BPL_DEBUG_SUSPEND false configure whether to suspend execution until a debugger has attached
[creator] $BPL_HEAP_DUMP_PATH write heap dumps on error to this path
[creator] $BPL_JAVA_NMT_ENABLED true enables Java Native Memory Tracking (NMT)
[creator] $BPL_JAVA_NMT_LEVEL summary configure level of NMT, summary or detail
[creator] $BPL_JFR_ARGS configure custom Java Flight Recording (JFR) arguments
[creator] $BPL_JFR_ENABLED false enables Java Flight Recording (JFR)
[creator] $BPL_JMX_ENABLED false enables Java Management Extensions (JMX)
[creator] $BPL_JMX_PORT 5000 configure the JMX port
[creator] $BPL_JVM_HEAD_ROOM 0 the headroom in memory calculation
[creator] $BPL_JVM_LOADED_CLASS_COUNT 35% of classes the number of loaded classes in memory calculation
[creator] $BPL_JVM_THREAD_COUNT 250 the number of threads in memory calculation
[creator] $JAVA_TOOL_OPTIONS the JVM launch flags
[creator] BellSoft Liberica JRE 17.0.1: Contributing to layer
[creator] Downloading from file:///platform/bindings/binary/bellsoft-jre17.0.1+12-linux-amd64.tar.gz
[creator] Verifying checksum
[creator] Expanding to /layers/paketo-buildpacks_bellsoft-liberica/jre
[creator] Adding 128 container CA certificates to JVM truststore
[creator] Writing env.launch/BPI_APPLICATION_PATH.default
[creator] Writing env.launch/BPI_JVM_CACERTS.default
[creator] Writing env.launch/BPI_JVM_CLASS_COUNT.default
[creator] Writing env.launch/BPI_JVM_SECURITY_PROVIDERS.default
[creator] Writing env.launch/JAVA_HOME.default
[creator] Writing env.launch/JAVA_TOOL_OPTIONS.append
[creator] Writing env.launch/JAVA_TOOL_OPTIONS.delim
[creator] Writing env.launch/MALLOC_ARENA_MAX.default
[creator] Launch Helper: Contributing to layer
[creator] Creating /layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/active-processor-count
[creator] Creating /layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/java-opts
[creator] Creating /layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/jvm-heap
[creator] Creating /layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/link-local-dns
[creator] Creating /layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/memory-calculator
[creator] Creating /layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/openssl-certificate-loader
[creator] Creating /layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/security-providers-configurer
[creator] Creating /layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/jmx
[creator] Creating /layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/jfr
[creator] Creating /layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/nmt
[creator] Creating /layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/security-providers-classpath-9
[creator] Creating /layers/paketo-buildpacks_bellsoft-liberica/helper/exec.d/debug-9
[creator] Java Security Properties: Contributing to layer
[creator] Writing env.launch/JAVA_SECURITY_PROPERTIES.default
[creator] Writing env.launch/JAVA_TOOL_OPTIONS.append
[creator] Writing env.launch/JAVA_TOOL_OPTIONS.delim
[creator]
[creator] Paketo Syft Buildpack 1.3.1
[creator] https://github.com/paketo-buildpacks/syft
[creator] Downloading from file:///platform/bindings/binary/syft_0.33.0_linux_amd64.tar.gz
[creator] Verifying checksum
[creator] Writing env.build/SYFT_CHECK_FOR_APP_UPDATE.default
[creator]
[creator] Paketo Executable JAR Buildpack 6.0.2
[creator] https://github.com/paketo-buildpacks/executable-jar
[creator] Class Path: Contributing to layer
[creator] Writing env/CLASSPATH.delim
[creator] Writing env/CLASSPATH.prepend
[creator] Process types:
[creator] executable-jar: java org.springframework.boot.loader.JarLauncher (direct)
[creator] task: java org.springframework.boot.loader.JarLauncher (direct)
[creator] web: java org.springframework.boot.loader.JarLauncher (direct)
[creator]
[creator] Paketo Spring Boot Buildpack 5.3.0
[creator] https://github.com/paketo-buildpacks/spring-boot
[creator] Creating slices from layers index
[creator] dependencies
[creator] spring-boot-loader
[creator] snapshot-dependencies
[creator] application
[creator] Launch Helper: Contributing to layer
[creator] Creating /layers/paketo-buildpacks_spring-boot/helper/exec.d/spring-cloud-bindings
[creator] Spring Cloud Bindings 1.8.0: Contributing to layer
[creator] Downloading from file:///platform/bindings/binary/spring-cloud-bindings-1.8.0.jar
[creator] Verifying checksum
[creator] Copying to /layers/paketo-buildpacks_spring-boot/spring-cloud-bindings
[creator] Web Application Type: Contributing to layer
[creator] Servlet web application detected
[creator] Writing env.launch/BPL_JVM_THREAD_COUNT.default
[creator] 4 application slices
[creator] Image labels:
[creator] org.springframework.boot.version
[creator]
[creator] Paketo Environment Variables Buildpack 4.0.1
[creator] https://github.com/paketo-buildpacks/environment-variables
[creator] Launch Configuration:
[creator] $BPE_<NAME> prepend value to $NAME, delimiting with OS path list separator
[creator] $BPE_APPEND_<NAME> append value to $NAME
[creator] $BPE_DEFAULT_<NAME> set default value for $NAME
[creator] $BPE_DELIM_<NAME> set delimeter to use when appending or prepending to $NAME
[creator] $BPE_OVERRIDE_<NAME> set $NAME to value
[creator] $BPE_PREPEND_<NAME> prepend value to $NAME
[creator] Environment Variables: Contributing to layer
[creator] Writing env.launch/JAVA_TOOL_OPTIONS.append
[creator] Writing env.launch/JAVA_TOOL_OPTIONS.delim
[creator] Writing env.launch/JAVA_TOOL_OPTIONS.prepend
[creator]
[creator] Paketo Image Labels Buildpack 4.0.2
[creator] https://github.com/paketo-buildpacks/image-labels
[creator] Build Configuration:
[creator] $BP_IMAGE_LABELS arbitrary image labels
[creator] $BP_OCI_AUTHORS Jazz the org.opencontainers.image.authors image label
[creator] $BP_OCI_CREATED the org.opencontainers.image.created image label
[creator] $BP_OCI_DESCRIPTION the org.opencontainers.image.description image label
[creator] $BP_OCI_DOCUMENTATION the org.opencontainers.image.documentation image label
[creator] $BP_OCI_LICENSES the org.opencontainers.image.licenses image label
[creator] $BP_OCI_REF_NAME the org.opencontainers.image.ref.name image label
[creator] $BP_OCI_REVISION the org.opencontainers.image.revision image label
[creator] $BP_OCI_SOURCE the org.opencontainers.image.revision image label
[creator] $BP_OCI_TITLE the org.opencontainers.image.title image label
[creator] $BP_OCI_URL the org.opencontainers.image.url image label
[creator] $BP_OCI_VENDOR the org.opencontainers.image.vendor image label
[creator] $BP_OCI_VERSION the org.opencontainers.image.version image label
[creator] Image labels:
[creator] org.opencontainers.image.authors
[creator] ===> EXPORTING
[creator] Adding layer 'paketo-buildpacks/ca-certificates:helper'
[creator] Adding layer 'paketo-buildpacks/bellsoft-liberica:helper'
[creator] Adding layer 'paketo-buildpacks/bellsoft-liberica:java-security-properties'
[creator] Adding layer 'paketo-buildpacks/bellsoft-liberica:jre'
[creator] Adding layer 'paketo-buildpacks/executable-jar:classpath'
[creator] Adding layer 'paketo-buildpacks/spring-boot:helper'
[creator] Adding layer 'paketo-buildpacks/spring-boot:spring-cloud-bindings'
[creator] Adding layer 'paketo-buildpacks/spring-boot:web-application-type'
[creator] Adding layer 'paketo-buildpacks/environment-variables:environment-variables'
[creator] Adding 5/5 app layer(s)
[creator] Adding layer 'launcher'
[creator] Adding layer 'config'
[creator] Adding layer 'process-types'
[creator] Adding label 'io.buildpacks.lifecycle.metadata'
[creator] Adding label 'io.buildpacks.build.metadata'
[creator] Adding label 'io.buildpacks.project.metadata'
[creator] Adding label 'org.springframework.boot.version'
[creator] Adding label 'org.opencontainers.image.authors'
[creator] Setting default process type 'web'
[creator] Saving registry.cn-shenzhen.aliyuncs.com/uid13/demo-app:1.0-SNAPSHOT...
[creator] *** Images (7e9822c9a502):
[creator] registry.cn-shenzhen.aliyuncs.com/uid13/demo-app:1.0-SNAPSHOT
[creator] Adding cache layer 'paketo-buildpacks/syft:syft'
Successfully built image 'registry.cn-shenzhen.aliyuncs.com/uid13/demo-app:1.0-SNAPSHOT'
> Pushing image 'registry.cn-shenzhen.aliyuncs.com/uid13/demo-app:1.0-SNAPSHOT' ..................................................
> Pushed image 'registry.cn-shenzhen.aliyuncs.com/uid13/demo-app:1.0-SNAPSHOT'
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/8.0/userguide/command_line_interface.html#sec:command_line_warnings
BUILD SUCCESSFUL in 58s
11 actionable tasks: 1 executed, 10 up-to-date
14:35:27: Execution finished 'bootBuildImage'.
Spring Boot Helm 部署 K8S
helm create
分别在 demo-app 和 demo-infra 目录下,执行 helm create svc
修改示例 demo-app svc/values.yaml
# 修改部分1:
image:
repository: registry.cn-shenzhen.aliyuncs.com/uid13/demo-app
pullPolicy: Always # 默认 IfNotPresent
# Overrides the image tag whose default is the chart appVersion.
tag: "1.0-SNAPSHOT"
# 修改部分2:
service:
type: ClusterIP
port: 8888 # demo-app server.port: 8888 对应
修改示例 demo-app svc/Chart.yaml
appVersion: "1.0-SNAPSHOT"
helm install
helm install demo-app svc
helm install demo-infra svc
helm list
PS C:\Users\Jazz\Desktop> helm list
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
demo-app default 1 2023-05-21 15:57:28.2823437 +0800 CST deployed svc-0.1.0 1.0-SNAPSHOT
demo-infra default 1 2023-05-21 15:53:20.3005265 +0800 CST deployed svc-0.1.0 1.0-SNAPSHOT
my-mysql-cluster default 1 2023-05-16 16:44:30.296044 +0800 CST deployed mysql-9.9.1 8.0.33
my-redis default 1 2023-05-16 16:44:12.3578441 +0800 CST deployed redis-17.10.3 7.0.11
my-zookeeper default 1 2023-05-16 16:43:55.5003718 +0800 CST deployed zookeeper-11.3.2 3.8.1
手动扩容 demo-app
PS C:\Users\Jazz\Desktop> kubectl scale deploy/demo-app-svc --replicas=2
deployment.apps/demo-app-svc scaled
kubectl get all
PS C:\Users\Jazz\Desktop> kubectl get all
NAME READY STATUS RESTARTS AGE
pod/demo-app-svc-789867d5f6-2vmgx 1/1 Running 0 10m
pod/demo-app-svc-789867d5f6-6ss5p 1/1 Running 0 2m29s
pod/demo-infra-svc-5568dc7f8f-fkc64 1/1 Running 0 14m
pod/my-mysql-cluster-primary-0 1/1 Running 9 (29m ago) 4d23h
pod/my-mysql-cluster-secondary-0 1/1 Running 8 (32m ago) 4d23h
pod/my-redis-master-0 1/1 Running 9 (32m ago) 4d23h
pod/my-zookeeper-0 1/1 Running 9 (32m ago) 4d23h
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/demo-app-svc ClusterIP 10.96.14.60 <none> 8888/TCP 10m
service/demo-infra-svc ClusterIP 10.96.214.229 <none> 8887/TCP 14m
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 4d23h
service/my-mysql-cluster-primary NodePort 10.96.25.228 <none> 3306:30002/TCP 4d23h
service/my-mysql-cluster-primary-headless ClusterIP None <none> 3306/TCP 4d23h
service/my-mysql-cluster-secondary NodePort 10.96.151.166 <none> 3306:30003/TCP 4d23h
service/my-mysql-cluster-secondary-headless ClusterIP None <none> 3306/TCP 4d23h
service/my-redis-headless ClusterIP None <none> 6379/TCP 4d23h
service/my-redis-master NodePort 10.96.209.202 <none> 6379:30005/TCP 4d23h
service/my-zookeeper NodePort 10.96.54.72 <none> 2181:30001/TCP,2888:32042/TCP,3888:32703/TCP 4d23h
service/my-zookeeper-headless ClusterIP None <none> 2181/TCP,2888/TCP,3888/TCP 4d23h
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/demo-app-svc 2/2 2 2 10m
deployment.apps/demo-infra-svc 1/1 1 1 14m
NAME DESIRED CURRENT READY AGE
replicaset.apps/demo-app-svc-789867d5f6 2 2 2 10m
replicaset.apps/demo-infra-svc-5568dc7f8f 1 1 1 14m
NAME READY AGE
statefulset.apps/my-mysql-cluster-primary 1/1 4d23h
statefulset.apps/my-mysql-cluster-secondary 1/1 4d23h
statefulset.apps/my-redis-master 1/1 4d23h
statefulset.apps/my-zookeeper 1/1 4d23h