ELK6.0以上版本----------filebeat多路径日志，通过redis发送logstash解析多索引模式

一、前言
本篇环境
Elasticsearch6.0 , Logstash6.0 , Kibana6.0 , filebeat6.0
这种索引模式建多了，比较吃内存请注意自己服务器的消耗
流程filebeat ⇒ redis ==>Logstash
流程图 请参考：https://blog.csdn.net/liyyzz33/article/details/86701949

二、目的
filebeat多日志传输，经Logstash过滤后输出多索引。

三、配置文件
filebeat.yml

主要通过log_type来判断

filebeat.prospectors:
- input_type: log
  paths:
    - /data/logs/php/*.log
  fields:
    log_source: php
    log_type: php
  tail_files: true
  scan_frequency: 60s
  backoff: 10s

- input_type: log
  paths:
    - /data/logs/web/*.log
  fields:
    log_source: web
    log_type: web
  tail_files: true
  scan_frequency: 60s
  backoff: 10s




output.redis:
      hosts: ["192.168.1.123"]
      port: 6379
      key: "filebeat"
      db: 0 
      timeout: 5
  
  

Logstash
input-output.conf

input {
  redis {
	data_type => "list"
    key => "filebeat"
    host => "192.168.1.123"
    port => 6379
    threads => 5
    codec => "json"
  }
}

filter {
}

output {
	if [fields][log_type] == "php" {
	  elasticsearch {	  
		hosts => ["192.168.1.60:9200","192.168.1.61:9200","192.168.1.62:9200"]
		index => "php-%{+YYYY.MM.dd}"
		action   =>  "index"
	  }
	}
	if [fields][log_type] == "web" {
	  elasticsearch {	  
		hosts => ["192.168.1.60:9200","192.168.1.61:9200","192.168.1.62:9200"]
		index => "web-%{+YYYY.MM.dd}"
		action   =>  "index"
	  }
	


  stdout {
  }
}

效果