日志filebeat-kafka

 架构图如上

【kafka端】

1.启动kafa 

./kafka-server-start.sh ../config/server.properties 1>/dev/null 2>&1 &     # kafka-server-start.sh 脚本在 kafka_2.12-2.2.0/bin路径下  

2.查看topic

 ./bin/kafka-topics.sh --list --zookeeper localhost:2181 | grep flume_log

3.新建topic

./bin/kafka-console-producer.sh --broker-list localhost:9092 --topic flume_log

【filebeat端】

1.编辑文件vim filebeat.yml

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /data/log/purchase/user_purchase_log_*
#=========================== user_purchase_log 商品访问日志 h5+buz =============================
##------------------------------kafka-----------------------------------
output.kafka:
  hosts: ["IP:9092"]
  topic: flume_log
  keep_alive: 10s

    2.运行：

 ./filebeat -e -c filebeat.yml

 [报错]
 在运行./filebeat时报出error错误：

Exiting: error loading config file: yaml: line 197: did not find expected key

Exiting: error loading config file: yaml: line 197: did not find expected key

发现是格式错误，空格多了，去除点多余的空格即可

【运行完之后可以再kafak端查看topic有没有数据】
 ./kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic flume_log --from-beginning