springboot接入elk日志平台

1 ELK简介

ELK是三个开源软件的缩写，分别表示：Elasticsearch , Logstash, Kibana 。现在新增了一个轻量级的日志收集处理工具FileBeat，Filebeat占用资源少，适合于在各个服务器上搜集日志后传输给Logstash，官方也推荐此工具。

2 应用架构图

springboot应用可以使用TCP连接将日志传输至logstash中，处理后输入es集群中待查询。

3 项目配置

3.1 引入依赖

<dependency>
		<groupId>net.logstash.logback</groupId>
		<artifactId>logstash-logback-encoder</artifactId>
		<version>5.3</version>
</dependency>
<dependency>
		<groupId>ch.qos.logback</groupId>
		<artifactId>logback-core</artifactId>
		<version>1.2.3</version>
</dependency>
<dependency>
		<groupId>ch.qos.logback</groupId>
		<artifactId>logback-classic</artifactId>
		<version>1.2.3</version>
</dependency>
<dependency>
		<groupId>ch.qos.logback</groupId>
		<artifactId>logback-access</artifactId>
		<version>1.2.3</version>
</dependency>

3.2 增加应用配置项

在application.properties文件增加如下配置，标识日志的项目来源。

spring.application.name=xxx      #项目名称

在application-env.properties文件增加如下配置，接收日志的服务器地址

logstash.host=10.253.128.4:4567,10.253.128.5:4567

3.3 修改日志配置信息

在logback-spring.xml中增加如下配置

<springProperty scope="context" name="logstashHost" source="logstash.host" defaultValue="127.0.0.1:4567"/>
<springProperty scope="context" name="appname" source="spring.application.name" defaultValue="xxx"/> #项目名称

<appender name="LOGSTASH" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
    <destination>${logstashHost}</destination>
    <encoder charset="UTF-8" class="net.logstash.logback.encoder.LogstashEncoder">
        <includeCallerData>true</includeCallerData>
        <!--Spring Cloud Sleuth 使用-->
        <includeMdcKeyName>traceId</includeMdcKeyName>
        <includeMdcKeyName>spanId</includeMdcKeyName>
        <includeMdcKeyName>spanExportable</includeMdcKeyName>
    </encoder>
    <keepAliveDuration>5 minutes</keepAliveDuration>
    <connectionStrategy> <roundRobin> <connectionTTL>5 minutes</connectionTTL> </roundRobin> </connectionStrategy>
    <reconnectionDelay>1 second</reconnectionDelay>
    <writeBufferSize>16384</writeBufferSize>
    <queueSize>1048576</queueSize>
</appender>

<springProfile name="prod">
    <root level="info">
         <appender-ref ref="LOGSTASH" /> #引用
    </root>
</springProfile>

3.4 loagstash配置

配置好TCP监听即可接收到应用日志

input {
    tcp {
        port => 4567
        codec => json_lines
    }
}