https://segmentfault.com/a/1190000016591126
https://elasticsearch.cn/question/8038
https://www.qedev.com/linux/317121.html。
docker构建ELK。https://hub.docker.com/r/sebp/elk
docker-compose.yml
elk:
image: sebp/elk
ports:
- "5601:5601"
- "9200:9200"
- "5044:5044"
如需使用filebeat发送日志。需要进入启动elk镜像后的容器,将 /etc/pki/tls/certs/logstash-beats.crt 拷贝到宿主机,然后配置filebeat时需要指定该文件。
该镜像产生3类日志:
/var/log/kibana/kibana5.log
/var/log/elasticsearch/elasticsearch.log
/var/log/logstash/logstasg-plain.log
遇到的坑。
Filebeat
fibeat6.3之后Filebeat prospectors 重命名到 inputs。
filebeat.yml
output:
logstash:
hosts:
- 127.0.0.1:5044
timeout: 15
ssl:
certificate_authorities:
- /etc/filebeat/logstash-beats.crt
filebeat.inputs:
- type: log
paths:
- /var/log/system.log
- /var/log/wifi.log
- type: log
paths:
- "/var/log/apache2/*"
fields:
apache: true
fields_under_root: true
enabled: true
processors:
- drop_fields:
fields: ["agent","ecs","input","agent"]
https://blog.csdn.net/yk20091201/article/details/90768632
filebeat除了发送日志,还有一些内置字段,需要移除的话 添加drop_fields
logstash.conf
input {
beats {
port => 5044
ssl => true
ssl_certificate => "/etc/pki/tls/certs/logstash-beats.crt"
ssl_key => "/etc/pki/tls/private/logstash-beats.key"
}
}
filter {
grok {
match => ["message","\[%{HTTPDATE:request_time}\]"]
}
grok {
match => {"log" => ["\/(?<app.name>[\w|-]+)\."]}
}
grok {
match => {"message" => "%{LOGLEVEL:log.level}"}
}
date {
locale => "zh"
match => ["request_time","yyyy-MM-dd HH:mm:ss Z"]
target => "@timestamp"
}
}
output {
elasticsearch {
hosts => ["localhost:9200"]
index => "applogs-%{+YYYY.MM.dd}"
}
stdout { codec => rubydebug }
}