ELK可以说是当前对分布式服务器集群日志做汇总、分析、统计和检索操作的很好的一套系统了。而Spring Boot作为一套为微服务而生的框架,自然也免不了处理分布式日志的问题,通过ELK日志系统来处理日志还是很有意义的。在这套系统中,E即为ElasticSearch,负责日志存储;L为LogStash,负责日志收集,并将日志信息写入ElasticSearch,K则为Kibana,负责将ElasticSearch中的日志数据进行可视化及分析检索操作。可以说将Spring Boot与ELK整合很大程度上相当于将Spring Boot与Logstash进行整合。将那么如何将Spring Boot与LogStash整合起来呢?
在Spring Boot当中,默认使用logback进行log操作。和其他日志工具如log4j一样,logback支持将日志数据通过提供IP地址、端口号,以Socket的方式远程发送。在Spring Boot中,通常使用logback-spring.xml来进行logback配置。
要想将logback与Logstash整合,必须引入logstash-logback-encoder包。该包的依赖如下:
<dependency>
<groupId>net.logstash.logback</groupId>
<artifactId>logstash-logback-encoder</artifactId>
<version>4.9</version>
</dependency>
将依赖添加好之后,就可以进行logback的配置了。logback-spring.xml配置如下:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE configuration>
<configuration>
<appender name="LOGSTASH" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
<destination>192.168.226.132:4560</destination>
<encoder charset="UTF-8" class="net.logstash.logback.encoder.LogstashEncoder" />
</appender>
<include resource="org/springframework/boot/logging/logback/base.xml"/>
<root level="INFO">
<appender-ref ref="LOGSTASH" />
<appender-ref ref="CONSOLE" />
</root>
</configuration>
logback的基础配置不多说,百度谷歌上一大把,这里我们只讨论与Logstash相关的部分。在上面的XML文件中的第4行至第7行即为Logstash的配置。destination为日志的发送地址,在配置Logstash的时候选择监听这个地址即可进行日志收集。这里我把Logstash部署在了本地的虚拟机上(其地址就是192.168.226.132),端口为4560。当然这里你需要把地址改成你自己的。下面的encoder是必选项。第9行到第14行则是为了保证Spring Boot原始的日志配置不被覆盖。这里logback配置完毕。
Logstash方面,在安装目录下新建一个文件夹conf,在conf文件夹下创建文件logstash-springboot.conf,内容如下:
input {
tcp {
port => 4560 //从本地的4560端口取日志。这里笔者将Logstash部署在了虚拟机192.168.226.132上,所以取的是本地地址。
codec => json_lines //需要安装logstash-codec-json_lines插件
}
}
output{
elasticsearch { hosts => ["localhost:9200"] } //输出到ElasticSearch
stdout { codec => rubydebug } //若不需要在控制台中输出,此行可以删除
}
如果你的Logstash没有安装logstash-codec-json_lines插件,通过以下命令安装:
bin/plugin install logstash-codec-json_lines
大功告成!最终效果如下:
-
{
-
"@timestamp" => "2017-04-23T06:14:52.734Z",
-
"@version" => 1,
-
"message" => "Tomcat started on port(s): 8000 (http)",
-
"logger_name" => "org.springframework.boot.context.embedded.tomcat.TomcatEmbeddedServletContainer",
-
"thread_name" => "main",
-
"level" => "INFO",
-
"level_value" => 20000,
-
"HOSTNAME" => "bogon",
-
"host" => "192.168.226.1",
-
"port" => 56168
-
}
-
{
-
"@timestamp" => "2017-04-23T06:14:52.744Z",
-
"@version" => 1,
-
"message" => "Initialize complete! I am init runner.",
-
"logger_name" => "org.fhp.springbootdemo.controller.InitRunner",
-
"thread_name" => "main",
-
"level" => "INFO",
-
"level_value" => 20000,
-
"HOSTNAME" => "bogon",
-
"host" => "192.168.226.1",
-
"port" => 56168
-
}
-
{
-
"@timestamp" => "2017-04-23T06:14:52.749Z",
-
"@version" => 1,
-
"message" => "MDC complete!IP=bogon",
-
"logger_name" => "org.fhp.springbootdemo.controller.InitRunner",
-
"thread_name" => "main",
-
"level" => "INFO",
-
"level_value" => 20000,
-
"HOSTNAME" => "bogon",
-
"ip" => "bogon",
-
"host" => "192.168.226.1",
-
"port" => 56168
-
}
参考:https://github.com/logstash/logstash-logback-encoder