微服务架构师封神之路02-为你的微服务应用添加日志
关于Kubernetes日志架构
讨论这个话题的技术文章不少,就是废话太多。简单说两个层面,
- 每个微服务内部:将日志写入到标准输出流中(Standard output stream, stdout and stderr)
- 整个Kubernetes cluster:在每个node上部署logging agent (运行在pod上的应用),将微服务写入stdout和stderr的日志转发到另外一个集中的地方存储。实现集中监控管理。
以上是一种最简单的日志架构设计,在实际操作中根据具体的情况还可能会有各种的变异。比如,微服务中的日志已经以文件形式存在,logging agent可能要先将file转化为stdout,再做转发操作。但核心思想是一样的,把分布在各处的日志集中到一个地方实现集中管理和监控。
我们的目标
为我们的微服务添加日志就包含了两个层面的工作,
- 为微服务项目添加具体的java日志工具,并将输出到stdout
- 在kubernetes node上安装logging agent,和转发后的data storage
紧接着就涉及了技术选型的问题, - 第一个问题我选择log4j2。原因也不多说了,目前log4j2优势明显,异步试日志的效率更是一骑绝尘,竟然可以达到其它工具的十倍效率以上。
- 第二个问题我选择Elasticsearch+Kibana。在Kubernetes文档上还有另一个选择是Stackdriver logging,看起来是在Google cloud platform上才能用,没有Elasticsearch+Kibana那么通用。
***今天我只能完成第一个目标,为helloworld项目添加日志工具,并输出到stdout。***第二个目标留着下次完成,我还要需要阅读更多的文档再实践。
helloworld project
项目结构
文件列表:
- pom.xml
- Dockerfile
- AppMain.java
- log4j2.xml
pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.2.4.RELEASE</version>
</parent>
<groupId>com.b5wang.cloudlab</groupId>
<artifactId>helloworld</artifactId>
<version>1.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>helloworld</name>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
<!-- dependencies' version -->
<spring-boot-dependencies.version>2.3.0.RELEASE</spring-boot-dependencies.version>
</properties>
<build>
<finalName>${project.artifactId}</finalName>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
<!-- =========================================================================
== Copy docker config files into target/docker-build folder ==
========================================================================= -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-resources-plugin</artifactId>
<version>3.1.0</version>
<executions>
<execution>
<id>generate-docker-build</id>
<phase>generate-resources</phase>
<goals>
<goal>copy-resources</goal>
</goals>
<configuration>
<outputDirectory>${project.build.directory}/docker-build</outputDirectory>
<resources>
<resource>
<directory>${project.basedir}/src/docker</directory>
<filtering>true</filtering>
</resource>
</resources>
</configuration>
</execution>
</executions>
</plugin>
<!-- =========================================================================
== Create docker image ==
========================================================================= -->
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.6.0</version>
<executions>
<execution>
<id>exec-docker-build</id>
<phase>package</phase>
<goals>
<goal>exec</goal>
</goals>
<configuration>
<executable>docker</executable>
<commandlineArgs>build --no-cache --tag b5wang/helloworld:1.1 -f ${project.build.directory}/docker-build/Dockerfile ${project.build.directory}</commandlineArgs>
</configuration>
</execution>
</executions>
</plugin>
<!-- fix error java.lang.ClassNotFoundException: org.apache.maven.doxia.siterenderer.DocumentContent -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-site-plugin</artifactId>
<version>3.8.2</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-project-info-reports-plugin</artifactId>
<version>3.0.0</version>
</plugin>
</plugins>
</build>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-dependencies</artifactId>
<version>${spring-boot-dependencies.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-log4j2</artifactId>
</dependency>
</dependencies>
</project>
Dockerfile
FROM openjdk:8-jdk-alpine
COPY helloworld.jar /app/helloworld.jar
ENTRYPOINT ["java","-jar","/app/helloworld.jar"]
AppMain.java
package com.b5wang.cloudlab.helloworld;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
@RestController
@EnableAutoConfiguration
public class AppMain {
private static final Logger LOGGER = LogManager.getLogger(AppMain.class);
@RequestMapping("/")
String home() {
LOGGER.trace("helloworld is called?!");
LOGGER.debug("helloworld is called?!");
LOGGER.info("helloworld is called?!");
LOGGER.error("helloworld is called?!");
LOGGER.fatal("helloworld is called?!");
return "minikube: Hello World!";
}
public static void main(String[] args) throws Exception {
LOGGER.info("helloworld app is starting......");// Log will not be printed out before spring app start.
SpringApplication.run(AppMain.class, args);
LOGGER.trace("helloworld app has started?!");
LOGGER.debug("helloworld app has started?!");
LOGGER.info("helloworld app has started?!");
LOGGER.error("helloworld app has started?!");
LOGGER.fatal("helloworld app has started?!");
}
}
log4j2.xml
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
<Appenders>
<Console name="STDOUT" target="SYSTEM_OUT">
<PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"/>
</Console>
</Appenders>
<Loggers>
<Logger name="com.b5wang.cloudlab.helloworld" level="debug" />
<Root level="error">
<AppenderRef ref="STDOUT"/>
</Root>
</Loggers>
</Configuration>
部署测试
具体过程如果需要参考请见:微服务架构师封神之路01-利用minikube部署一个最简单的应用
查看pod日志的几种方式:
- Minikube UI: $ minikube dashboard
- kubectl logs -f helloworld
- docker service logs
今天就到这里!
下次我们讨论如何将微服务应用和cloud platform同意起来,形成一个完整的日志架构。谢谢,再见! :)
参考材料与延伸阅读
Kubernetes logging architecture: https://kubernetes.io/docs/concepts/cluster-administration/logging/
kubectl log command - https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#logs
Docker logging: https://docs.docker.com/config/containers/logging/configure/
Log4j2 logging in the cloud: https://logging.apache.org/log4j/2.x/manual/cloud.html
Logs: https://12factor.net/logs