版本
- k8s版本:1.24.4(k8s版本不重要,高于1.15就可以)
- SpringCloud版本:2021.0.3(如果网关使用SpringGateway的话会很重要)
- Skywalking版本:8.7.0(本文仅针对8.7.0版本,因为每一版本差异都比较大)
- ElasticSearch7
- Centos7
流程原理
Skywalking可以划分为三个大模块:
- SkyWalking Agent:作为运行时的代理端,可以实现无侵入式对调用链的数据进行采集。
- SkyWalking OAP:作为链路系统中心具备收集、传输等功能。
- SkyWalking UI:ui管理界面
- ElasticSearch:数据存储中心
SkyWalking服务搭建
新建文件夹
mkdir /opt/skywalking
mkdir /opt/skywalking/docker
mkdir /opt/skywalking/k8s
新建构造文件
创建skywalking-oap构造文件
vim /opt/skywalking/k8s/skywalking-oap.yml
skywalking-oap.yaml内容如下:
apiVersion: apps/v1
kind: Deployment
metadata:
name: oap
namespace: skywalking
spec:
replicas: 1
selector:
matchLabels:
app: oap
release: skywalking
template:
metadata:
labels:
app: oap
release: skywalking
spec:
containers:
- name: oap
image: apache/skywalking-oap-server:8.7.0-es7
env:
- name: TZ
value: Asia/Shanghai
# 选择ES作为存储器
- name: SW_STORAGE
value: elasticsearch7
# ES的服务节点
- name: SW_STORAGE_ES_CLUSTER_NODES
value: 127.0.0.1:9200
# ES的账号(如果ES布设账号密码,那么可以不填写)
- name: SW_ES_USER
value: my-elastic
# ES的账号(如果ES布设账号密码,那么可以不填写)
- name: SW_ES_PASSWORD
value: 123456
imagePullPolicy: IfNotPresent
ports:
- containerPort: 11800
name: grpc
- containerPort: 12800
name: rest
---
apiVersion: v1
kind: Service
metadata:
name: oap
namespace: skywalking
labels:
service: oap
spec:
ports:
- port: 12800
name: rest
nodePort: 31280
- port: 11800
name: grpc
nodePort: 31180
#type: ClusterIP
# 为了测试方便这里暴露为主机端口,生产环境请使用ClustreIP
type: NodePort
selector:
app: oap
创建skywalking-ui构造文件
vim /opt/skywalking/k8s/skywalking-ui.yml
skywalking-ui.yml内容如下
apiVersion: apps/v1
kind: Deployment
metadata:
name: ui-deployment
namespace: skywalking
labels:
app: ui
spec:
replicas: 1
selector:
matchLabels:
app: ui
template:
metadata:
labels:
app: ui
spec:
containers:
- name: ui
image: apache/skywalking-ui:8.7.0
ports:
- containerPort: 8080
name: page
env:
- name: TZ
value: Asia/Shanghai
- name: SW_OAP_ADDRESS
value: http://oap:12800
# volumeMounts:
# - mountPath: /etc/localtime
# name: volume-localtime
# volumes:
# - hostPath:
# path: /etc/localtime
# type: ''
# name: volume-localtime
---
apiVersion: v1
kind: Service
metadata:
name: ui
namespace: skywalking
labels:
service: ui
spec:
ports:
- port: 8080
name: page
nodePort: 31234
type: NodePort
selector:
app: ui
发布运行
cd /opt/skywalking/k8s
创建命名空间
kubectl create ns skywalking
发布oap、ui服务
kubectl apply -f .
等待服务创建起来后,执行以下命令进行检查
kubectl get all -n skywalking
构建Skywalking Sidecar镜像
cd /opt/skywalking/docker
wget https://archive.apache.org/dist/skywalking/8.7.0/apache-skywalking-apm-8.7.0.tar.gz
tar -zxvf apache-skywalking-apm-8.7.0.tar.gz
这个版本默认是不支持SpringGateway的,所以需要手动添加gateway的支持jar包,执行一下下命令(把解压出来的包的option-plugins的gateway移动到plugins下即可)。
mv apache-skywalking-apm-bin/agent/optional-plugins/apm-spring-cloud-gateway-3.x-plugin-8.7.0.jar apache-skywalking-apm-bin/agent/plugins/
构建agent的Docker镜像
vim /opt/skywalking/docker/Dockerfile
Dockerfile内容如下
FROM busybox:latest
ENV LANG=C.UTF-8
RUN set -eux && mkdir -p /usr/skywalking/agent/
ADD apache-skywalking-apm-bin/agent/ /usr/skywalking/agent/
WORKDIR /
构建docker镜像
cd /opt/skywalking/docker
这里使用Harbor作为镜像仓库地址(因为要挂载到每个不同的微服务上执行,微服务部署在多个不同的主机节点上,所以需要发布到镜像中心库,这里使用Harbor,省事的也可以使用阿里云的镜像仓库,仓库的搭建使用这里不进行说明。)
docker build . -t harbor的Ip:端口号/plugins/skywalking-agent-sidecar:8.7.0
推送镜像
docker push harbor的Ip:端口号/plugins/skywalking-agent-sidecar:8.7.0
SpringCloud微服务k8s部署集成skywalking
在需要集成的每个SrpingBoot的应用上引入maven依赖:
<dependency>
<groupId>org.apache.skywalking</groupId>
<artifactId>apm-toolkit-log4j-2.x</artifactId>
<version>8.7.0</version>
</dependency>
在应用的/resources目录下创建log4j2-spring.xml文件:
log4j2-spring.xml内容如下:
<?xml version="1.0" encoding="UTF-8"?>
<!--日志级别以及优先级排序: OFF > FATAL > ERROR > WARN > INFO > DEBUG > TRACE > ALL -->
<!-- Configuration后面的status,这个用于设置log4j2自身内部的信息输出,可以不设置,当设置成trace时,
你会看到log4j2内部各种详细输出。可以设置成OFF(关闭) 或 Error(只输出错误信息)。
-->
<!--monitorInterval:Log4j能够自动检测修改配置 文件和重新配置本身,设置间隔秒数-->
<configuration status="WARN" monitorInterval="30">
<Properties>
<Property name="log.path">/usr/local/logs</Property>
<Property name="moduleName">my-application-name</Property>
<Property name="logging.swkj.pattern">
[%date{yyyy-MM-dd HH:mm:ss.SSS}][${moduleName}][%level][%traceId][%thread][%C][%M][%line][%X{X-B3-TraceId} %X{X-B3-SpanId} %X{X-B3-ParentSpanId}]-%m%n
</Property>
</Properties>
<Appenders>
<!-- 输出控制台日志的配置 -->
<Console name="Console" target="SYSTEM_OUT">
<!--控制台只输出level及以上级别的信息(onMatch),其他的直接拒绝(onMismatch)-->
<ThresholdFilter level="debug" onMatch="ACCEPT" onMismatch="DENY"/>
<!-- 输出日志的格式 -->
<PatternLayout pattern="${logging.swkj.pattern}"/>
</Console>
<RollingRandomAccessFile name="debugRollingFile" fileName="${log.path}/${moduleName}/${moduleName}.log"
filePattern="${log.path}/${moduleName}/${moduleName}_%d{yyyy-MM-dd}_%i.log">
<ThresholdFilter level="DEBUG" onMatch="ACCEPT" onMismatch="DENY"/>
<PatternLayout charset="UTF-8" pattern="${logging.swkj.pattern}"/>
<Policies>
<TimeBasedTriggeringPolicy interval="1"/>
<SizeBasedTriggeringPolicy size="100 MB"/>
</Policies>
<DefaultRolloverStrategy max="30"/>
</RollingRandomAccessFile>
<RollingRandomAccessFile name="errorRollingFile" fileName="${log.path}/${moduleName}/${moduleName}.log"
filePattern="${log.path}/${moduleName}/${moduleName}_%d{yyyy-MM-dd}_%i.log">
<ThresholdFilter level="ERROR" onMatch="ACCEPT" onMismatch="DENY"/>
<PatternLayout charset="UTF-8" pattern="${logging.swkj.pattern}"/>
<Policies>
<TimeBasedTriggeringPolicy interval="1"/>
<SizeBasedTriggeringPolicy size="100 MB"/>
</Policies>
<DefaultRolloverStrategy max="30"/>
</RollingRandomAccessFile>
<GRPCLogClientAppender name="grpc-log">
<PatternLayout pattern="${logging.swkj.pattern}"/>
</GRPCLogClientAppender>
</Appenders>
<Loggers>
<!-- ALL < TRACE < DEBUG < INFO < WARN < ERROR < FATAL < OFF -->
<!-- <Logger name="com.swkj.sys" level="debug"/>-->
<!-- <Logger name="org.apache.kafka" level="warn"/>-->
<Root level="info">
<AppenderRef ref="Console"/>
<AppenderRef ref="debugRollingFile"/>
<AppenderRef ref="errorRollingFile"/>
<AppenderRef ref="grpc-log"/>
</Root>
</Loggers>
</configuration>
构建SpringBoot的deployment、service的yaml文件
mkdir /usr/project
cd /usr/project
kubectl create ns spring-app
vim my-boot-deploy.yml
my-boot-deploy.yml内容如下:
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-gateway
namespace: spring-app
labels:
name: my-gateway
spec:
replicas: 1
selector:
matchLabels:
name: my-gateway
template:
metadata:
labels:
name: my-gateway
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: env
operator: In
values:
- dev
initContainers:
- image: 'harbor的Ip:端口号/plugins/skywalking-agent-sidecar:8.7.0'
name: sw-agent-sidecar
imagePullPolicy: IfNotPresent
command:
- sh
args:
- '-c'
- >-
mkdir -p /skywalking/agent && cp -r /usr/skywalking/agent/*
/skywalking/agent
volumeMounts:
- name: sw-agent
mountPath: /skywalking/agent
containers:
- name: my-gateway
image: harbor的Ip:端口号/project/my-gateway:latest
volumeMounts:
- name: sw-agent
mountPath: /usr/skywalking/agent
command: ["sh","-c","java -javaagent:/usr/skywalking/agent/skywalking-agent.jar=agent.service_name=dev-gateway,collector.backend_service=oap.skywalking:11800,plugin.toolkit.log.grpc.reporter.server_host=oap.skywalking,plugin.toolkit.log.grpc.reporter.server_port=11800 -server -Xms256m -Xmx512m -Dfile.encoding=utf-8 -Djava.io.tmpdir=/var/tmp -Duser.timezone=Asia/Shanghai -jar my-gateway.jar --server.port=8080 --spring.profiles.active=dev"]
ports:
- containerPort: 8080
volumes:
- name: sw-agent
emptyDir: { }
构建boot应用的service文件
vim my-gateway-svc.yml
my-gateway-svc.yml内容如下:
apiVersion: v1
kind: Service
metadata:
name: my-gateway
namespace: spring-app
labels:
name: my-gateway
spec:
type: NodePort
ports:
- nodePort: 32000
targetPort: 8080
port: 8080
selector:
name: my-gateway
每个SpringBoot的应用都如上导入skywalk的agent,即可,然后观察启动日志中和下图一样即应用成功:
注意一点的是,skywalking-ui的浏览器缓存特别难缠,有时候构建成功之后需要到设置里面进行强制清理浏览缓存数据才会显示效果。
成功的skywalking效果图: