实验背景:
Kafka->Logstash->Elasticsearch实验环境安装完成;
Logstash的启动文件设置有过滤机制;
Logstash启动文件:
input {
kafka {
zk_connect => "10.10.16.252:2181,10.10.16.253:2181,10.10.16.249:2181"
group_id => "test-consumer-group"
topic_id => "MyPattern"
reset_beginning => false # boolean (optional), default: false
consumer_threads => 5 # number (optional), default: 1
decorate_events => true # boolean (optional), default: false
}
}
filter{
grok{
patterns_dir => "./patterns"
match => { "message" => ["#字段内容#"]}
}
}
output {
if "jsonparsefailure" not in [tags]{
elasticsearch{
host => "localhost"
}
}
}
发送数据的代码如下所示:
p