使用docker-compose与SpringBoot搭建ELK日志分析系统

最新推荐文章于 2024-07-02 00:09:27 发布

用java控制挖掘机炒菜

最新推荐文章于 2024-07-02 00:09:27 发布

阅读量865

点赞数

分类专栏： docker

docker 专栏收录该内容

13 篇文章 1 订阅

订阅专栏

使用docker-compose与SpringBoot搭建ELK日志分析系统

转载出： https://blog.csdn.net/huangliuyu00/article/details/84945268

结合自己的实际项目有部分内容改动

ELK对应Elasticsearch、Logstash、Kibana
Logstash作为日志采集工具，向Elasticsearch写日志信息；
Elasticsearch提供存储与检索功能；
Kibana为Elasticsearch的查询接口，提供友好的图形界面。

搭建ELK环境

这里使用docker-compose把ELK作为一组项目容器启动，这里请提前搭好docker、docker-compose环境。

新建 /data/elk 目录
在elk目录下创建文件 docker-compose配置文件
docker-compose.yml

version: '2'
services:
  elasticsearch:
    image: elasticsearch
    environment:
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    volumes:
      - $PWD/elasticsearch/data:/usr/share/elasticsearch/data
    hostname: elasticsearch
    restart: always
    ports:
      - "9200:9200"
      - "9300:9300"
  kibana:
    image: kibana
    environment:
      - ELASTICSEARCH_URL=http://elasticsearch:9200 #elasticsearch查询接口地址
    hostname: kibana
    depends_on:
      - elasticsearch  #后于elasticsearch启动
    restart: always
    ports:
      - "5601:5601"
  logstash:
    image: logstash
    command: logstash -f /etc/logstash/conf.d/logstash.conf  #logstash 启动时使用的配置文件
    volumes:
      - $PWD/logstash/conf.d:/etc/logstash/conf.d  #logstash 配文件位置
      - $PWD/logst:/tmp
    hostname: logstash
    restart: always
    depends_on:
      - elasticsearch  #后于elasticsearch启动
    ports:
      - "7001-7005:7001-7005"
      - "4560:4560"
      - "9600:9600"

创建logstash启动配置文件
/data/elk/logstash/conf.d/logstash.conf

input {
tcp {
mode => "server"
#host => "192.168.0.210"
port => 4560
codec => json_lines
tags => ["tags"]
}
}
output{
elasticsearch {
    hosts => ["192.168.0.210:9200"]

index => "%{[appname]}"
}
stdout {
codec => rubydebug
}
}

如果elk 3个镜像下不来，可以配置国内的镜像加速，如阿里的、docker中国官方的
/etc/docker/daemon.json
{
“registry-mirrors”: [“https://registry.docker-cn.com”]
}

启动、停止容器组

#/data/elk 目录下
## 启动
[root@localhost elk]# docker-compose up -d 
Creating network "elk_default" with the default driver
Creating elk_elasticsearch_1_62f42e83ab51 ... done
Creating elk_logstash_1_b4f112872efa      ... done
Creating elk_kibana_1_dd80d748dac1        ... done
[root@localhost elk]# docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                                                                              NAMES
e8715946efd3        kibana              "/docker-entrypoin..."   3 seconds ago       Up 2 seconds        0.0.0.0:5601->5601/tcp                                                             elk_kibana_1_c07ef0b34a97
0cfc13d30a68        logstash            "/docker-entrypoin..."   3 seconds ago       Up 2 seconds        0.0.0.0:4560->4560/tcp, 0.0.0.0:7001-7005->7001-7005/tcp, 0.0.0.0:9600->9600/tcp   elk_logstash_1_6abacb9c09ba
7bf21946fb20        elasticsearch       "/docker-entrypoin..."   4 seconds ago       Up 3 seconds        0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp                                     elk_elasticsearch_1_47fc541fb48d


## 停止容器组
[root@localhost elk]# docker-compose down
Stopping elk_kibana_1_c07ef0b34a97        ... done
Stopping elk_logstash_1_6abacb9c09ba      ... done
Stopping elk_elasticsearch_1_47fc541fb48d ... done
Removing elk_kibana_1_c07ef0b34a97        ... done
Removing elk_logstash_1_6abacb9c09ba      ... done
Removing elk_elasticsearch_1_47fc541fb48d ... done
Removing network elk_default
[root@localhost elk]#

配置SpringBoot应用向Logstash输入日志

（1）添加 logstash-logback 依赖包

<dependency>
      <groupId>net.logstash.logback</groupId>
      <artifactId>logstash-logback-encoder</artifactId>
      <version>4.9</version>
</dependency>

（2）在resource目录添加logback配置
logback.xml

<appender name="LOGSTASH"  class="net.logstash.logback.appender.LogstashTcpSocketAppender">
   <destination>192.168.0.210:4560</destination>
   <!-- encoder必须配置,有多种可选 -->
   <encoder charset="UTF-8"  class="net.logstash.logback.encoder.LogstashEncoder" >
      <!-- "appname":"longji-provider" 的作用是指定创建索引的名字时用，并且在生成的文档中会多了这个字段  -->
      <customFields>{"appname":"longji-provider"}</customFields>
   </encoder>
</appender>