蜂巢APP对接ELK统一日志

docker-compose的方式对接ELK

（1）一键部署docker和docker-compose环境

docker容器简介 （一键部署脚本）

#!/bin/bash

install docker shell

#step 1: 安装必要的一些系统工具
sudo yum install -y yum-utils device-mapper-persistent-data lvm2

2: 添加软件源信息

sudo yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

3: 更新并安装 Docker-CE

sudo yum makecache fast
sudo yum -y install docker-ce
#4: 开启Docker服务
sudo service docker start

4.添加镜像加速器

echo “4.添加镜像加速器…”
sudo tee /etc/docker/daemon.json <<-‘EOF’
{
“registry-mirrors”: [“https://xcweb0za.mirror.aliyuncs.com”]
}
EOF

5.重新启动服务

echo “5.重新启动服务…”
sudo systemctl daemon-reload
sudo systemctl restart docker

6.检测

echo “6.检测…”
docker info

7.安装docker-compose

echo “7.安装docker-compose”
curl -L https://get.daocloud.io/docker/compose/releases/download/1.22.0/docker-compose-uname -s-uname -m > /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose

8.验证是否安装成功

echo “8.验证是否安装成功…”
docker-compose -v

（2）编写docker-compose文件

version: ‘2’
services:
filebeat-logcenter:
image: harbor.qilu-dev.com/elk_public/filebeat:6.8.2
container_name: filebeat-logcenter
network_mode: host
restart: always
volumes:

• /root/log/filebeat.yml:/usr/share/filebeat/filebeat.yml:ro
• /usr/java/logs/:/usr/java/logs/:ro
• /usr/java/apache-tomcat-8.0.46/logs/:/usr/java/apache-tomcat-8.0.46/logs/:ro
  environment:
• TZ=Asia/Shanghai
• KAFKA_HOSTS=10.29.131.62:9092,10.29.131.63:9092,10.29.131.64:9092,10.29.131.65:9092

（3）filebeat模板文件

filebeat.inputs:

• type: log
  enabled: true
  paths:

• /usr/java/logs/ztapi.log
  fields:
  headers.appName: ztapi
  headers.key: ztapi
  fields_under_root: true
  multiline.pattern: ‘^[’
  multiline.negate: ‘true’
  multiline.match: ‘after’

• type: log
  enabled: true
  paths:

• /usr/java/apache-tomcat-8.0.46/logs/catalina.out
  fields:
  headers.appName: ztapitomcat
  headers.key: ztapitomcat
  fields_under_root: true
  multiline.pattern: ‘^\d{2}-\w{3}-\d{4}’
  multiline.negate: ‘true’
  multiline.match: ‘after’

• type: log
  enabled: true
  paths:

• /usr/java/apache-tomcat-8.0.46/logs/localhost_access_log*
  fields:
  headers.appName: ztapilocalhostaccess
  headers.key: ztapilocalhostaccess
  fields_under_root: true
  multiline.pattern: ‘^\d+.\d+.\d+.\d+’
  multiline.negate: ‘true’
  multiline.match: ‘after’
  filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

output.kafka:
enabled: true
hosts: “${KAFKA_HOSTS}”
topic: logCenter-other
key: appname
partition.hash:
reachable_only: false
keep_alive: 60
required_acks: 1

processors:

• add_host_metadata:
  cache.ttl: 5m
• rename:
  fields:
• from: “host.name”
  to: “headers.host”
• drop_fields:
  fields: [“metadata”, “log”, “input”, “ecs”, “agent”, “host”, “prospector”, “source”, “beat”, “offset”]

（3）一键启动对接

设置好相应目录文件

一键启动

docker-compose up -d