实验环境:
CentOS 7
1. 安装:
curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.5.4-x86_64.rpm sudo rpm -vi filebeat-6.5.4-x86_64.rpm
curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.5.4-linux-x86_64.tar.gz tar xzvf filebeat-6.5.4-linux-x86_64.tar.gz
2. 配置:
vim /etc/filebeat/filebeat.yml
# 添加需要跟踪的log
- type: log
enabled: true
paths:
- /root/test_elk/test_elk*.log
fields:
log_topic: test_elk
# 配置输出
# 可以输出到es,logstash,kibana
output.logstash:
hosts: ["192.168.3.51:5044"]
output.kafka:
# initial brokers for reading cluster metadata
hosts: ["kafka01:9092","kafka02:9092","kafka03:9092"]
# message topic selection + partitioning
partition.round_robin:
reachable_only: false
topic: '%{[fields.log_topic]}'
required_acks: 1
compression: gzip
max_message_bytes: 1000000
------------------------------------------------------------------------------------------
Load the index template in Elasticsearch
output.elasticsearch:
# Array of hosts to connect to.
hosts: ["es01:9200"]
index: "filebeat-test-%{+yyyy.MM.dd}"
# Optional protocol and basic auth credentials.
#protocol: "https"
#username: "elastic"
#password: "changeme"
https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-template.html
------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------
使用module
filebeat modules enable postgresql
vim /etc/filebeat/modules.d/postgresql.yml
- module: postgresql
# All logs
log:
enabled: true
var.paths: ["/usr/local/pgsql/data/log/*.log*"]
------------------------------------------------------------------------------------------
3. 启动
service filebeat start
4. kibana中配置。
登录到kibana。
1. 选择左边?列表中的 "Management" ,显示管理界面。
2. 选择Elasticsearch中的 "Index Management",显示出目前在ES中所有的 Index。
3. 回到上一级中的管理界面。(选择左边?列表中的 "Management" )
4. 选择Kibana中的"Index Management",进入到 "Index Patterns"
5. 创建一个新的 index pattern。(选择Create index pattern)
5.1 Step 1 选择要匹配的 "Index pattern"
5.2 Step 2 Configure settings,时间过滤字段名称。可以不选,也可以选
6. 选择左边?列表中的 "Discover",index pattern就会出现在选择框里了。