1 Filebeat介绍
轻量型日志采集器。当您要面对成百上千、甚至成千上万的服务器、虚拟机和容器生成的日志时,请告别 SSH 吧。Filebeat 将为您提供一种轻量型方法,用于转发和汇总日志与文件,让简单的事情不再繁杂。
2 安装
# wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.0.0-linux-x86_64.tar.gz
# tar zxf filebeat-6.0.0-linux-x86_64.tar.gz -C /Data/apps/
3 应用一:Filebeat采集数据,写入到logstash
3.1 修改Filebeat配置
101.1.21.170是logstash机器的ip。
# cd /Data/apps/filebeat-6.0.0/
# sudo vi filebeat.yml
filebeat.prospectors:
enabled: true
- /Data/apps/nginx/logs/access-filebeat-test.log
output.logstash:
hosts: ["101.1.21.170:5044"]
启动Filebeat
# sudo ./filebeat -e -c filebeat.yml
3.2 logstash配置
101.1.21.170是logstash机器的ip。
# /Data/apps/logstash-6.0.0/bin
# sudo vi filebeat.conf
input {
beats {
port => 5044
}
}
output {
stdout {
codec => "rubydebug"
}
}
启动logstash
# ./logstash -f filebeat.conf
3.3 向/Data/apps/nginx/logs/access-filebeat-test.log追加信息
# echo "CleverCode is coding" >> /Data/apps/nginx/logs/access-filebeat-test.log
4 应用二:Filebeat采集数据,写入到redis,然后logstash从reids队列中拉取数据
4.1 filebeat配置
# cd /Data/apps/filebeat-6.0.0/
# sudo vi filebeat.yml
filebeat.prospectors:
enabled: true
- /Data/apps/nginx/logs/access-filebeat-test.log
output.redis:
hosts: ["10.1.20.170"]
port: 6379
key: "filebeat"
db: 0
timeout: 5
启动Filebeat
# sudo ./filebeat -e -c filebeat.yml
4.2 logstash配置
# /Data/apps/logstash-6.0.0/bin
# sudo vi redis.conf
input {
redis {
host => "10.1.20.170"
port => 6379
key => "filebeat"
data_type => "list"
}
}
output {
stdout {
codec => rubydebug
}
}
启动logstash
# ./logstash -f redis.conf
4.3 向/Data/apps/nginx/logs/access-filebeat-test.log追加信息
# echo "this msg is from reids" >> /Data/apps/nginx/logs/access-filebeat-test.log
技术交流
CleverCode是一名架构师,技术交流,咨询问题,请加CleverCode创建的qq群(架构师俱乐部):517133582。加群和腾讯,阿里,百度,新浪等公司的架构师交流。【架构师俱乐部】宗旨:帮助你成长为架构师!