Kibana上查不到最近的log,看了filebeat的log,发现一个问题
2021-06-07T07:53:18.169Z INFO log/input.go:596 File is falling under ignore_older before harvesting is finished. Adjust your close_* settings: /mnt/disks/warehouse/data_warehouse/sdk_report/report/receive.2021-06-01.log
思考了下,最近修改了filebeat的配置,主要是要上报的log太多,不想要上报太久之前的数据,就设置了ignore_older参数,配置如下:
filebeat.inputs:
- type: log
enabled: true
paths:
- /mnt/disks/log_warehouse/roiquery_log
tail_files: true
multiline.pattern: ^\w+\s\[
multiline.negate: true
multiline.match: after
tags: ["api.roiquery"]
- type: log
enabled: true
paths:
- /mnt/disks/warehouse/data_warehouse/sdk_report/report/*.log
tail_files: true
ignore_older: 12h # ignore files which were modified more than 12 hours
close_inactive: 10m
tags: ["data_collector"]
output.logstash:
hosts: ["10.138.0.3:5044"]
proces