ELK版本:7.2
Zookeeper:3.4.14/3.5.5
问题描述:
2019-07-30T11:17:12,834][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch.
{:status=>400, :action=>["index", {:_id=>nil, :_index=> "metricbeat-7.2.0-2019.07.30",
:_type=>"_doc", :routing=>nil}, #<LogStash::Event:0x65025a7f>], :response=>{"index"=>{
"_index"=>"metricbeat-7.2.0-2019.07.30", "_type"=>"_doc", "_id"=>"EyzhQGwBQV56mUOtwXCt",
"status"=>400, "error"=>{"type"=>"mapper_parsing_exception",
"reason"=>"failed to parse field [zookeeper.server.version_date] of type [date]
in document with id 'EyzhQGwBQV56mUOtwXCt'", "caused_by"=>{"type"=>"illegal_argument_exception",
"reason"=>"failed to parse date field [09/30/2012 17:52 GMT] with format [strict_date_optional_time||epoch_millis]",
"caused_by"=>{"type"=>"date_time_parse_exception", "reason"=>"Failed to parse with all enclosed parsers"}}}}}}
问题分析
1、产生该问题的原因是zookeeper.server.version_date
被当做日期来进行解析,但是09/30/2012 17:52 GMT
无法被正确的解析导致此问题
2、起初以为是zookeeper版本问题,但将 zookeeper升级到最新版本3.5.5后仍存在此问题。
解决办法:
修改modules.d/zookeeper.yml
配置文件,在metricbeat将监控结果发送到logstash 或 elasticsearch前,过滤列zookeeper.server.version_date
,修改后的配置文件如下:
# Module: zookeeper
# Docs: https://www.elastic.co/guide/en/beats/metricbeat/7.2/metricbeat-module-zookeeper.html
- module: zookeeper
#metricsets:
# - mntr
# - server
period: 10s
hosts: ["192.168.1.2:2181"]
processors:
- drop_fields:
fields: ["zookeeper.server.version_date"]
显示效果如下:
–END–