配置filebeat-6.6.1

1.首先从官网下载

wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.6.1-linux-x86_64.tar.gz

2.解压文件

tar -zxf filebeat-6.6.1-linux-x86_64.tar.gz

3.配置文件shipper.yml,这个是启动的时候指定的文件

filebeat.config.inputs:
  path: config/*.yml
  reload.enabled: false

processors:
  - drop_fields:
      fields: ["input_type", "beat", "offset"]

output.kafka:
    version: 2.0.0
    hosts: ["172.18.2.139:9092","172.18.2.140:9092","172.18.2.141:9092"]
    topic: '%{[type]}'
    partition.round_robin:
      reachable_only: true

logging.level: info
logging.to_files: true
logging.files:
  path: /mnt/logs/filebeat
  name: filebeat
  rotateeverybytes: 10485760  #等于10MB
  keepfiles: 7
  permissions: 0600

 

在/usr/local/filebeat-6.6.1-linux-x86_64/config

配置nginx

[root@nginxae-wan-01 config]# cat nginx.yml
- type: log
  paths:
    - /mnt/logs/nginx/access/*.log
  fields:
    type: nginxaccess
    host: nginxae-wan-01
  fields_under_root: true
  exclude_lines: ['favicon.ico']
  close_inactive: 12h
  ignore_older: 24h
  clean_inactive: 48h

- input_type: log
  paths:
    - /mnt/logs/nginx/error/*.log
  fields:
    type: nginxerror
    host: nginxae-wan-01
  fields_under_root: true
  exclude_lines: ['favicon.ico']
  close_inactive: 12h
  ignore_older: 24h
  clean_inactive: 48h

5.启动程序

cd /usr/local/filebeat-6.6.1-linux-x86_64 && nohup ./filebeat -c shipper.yml >/dev/null 2>&1 &

 

配置收集的log传到kafaka上去，这样kafaka在把数据传输到logstash，传给elsearch。

这样就可以去kibana面板查看了。