Elasticsearch集群启动
启动es集群
Filebeat 配置及启动
创建配置文件log-monitor.yml内容如下:
[root@localhost filebeat]# vi log-monitor.yml
filebeat.inputs:
- type: log
enabled: true
paths:
- /opt/apps/filebeat/logs/*.log
setup.template.settings:
index.number_of_shards: 3
output.logstash:
hosts: ["192.168.12.10:5044"]```
启动 filebeat服务
[root@localhost filebeat]# ./filebeat -e -c log-monitor.yml
Logstash 配置及启动
创建配置文件log-moni.conf内容如下:
[root@localhost logstash]# vi log-moni.conf
input {
beats {
port => "5044"
}
}
filter {
mutate {
split => {"message"=>"|"}
}
mutate {
add_field => {
"userId" => "%{message[1]}"
"visit" => "%{message[2]}"
"date" => "%{message[3]}"
}
}
mutate {
convert => {
"userId" => "integer"
"visit" => "string"
"date" => "string"
}
}
}
output {
elasticsearch {
hosts => ["192.168.12.10:9200","192.168.12.11:9200","192.168.12.12:9200"]
}
}
启动Logstash服务
[root@localhost logstash]# ./bin/logstash -f log-moni.conf
模拟写入数据到日志文件
[root@localhost logs]# vi moni.log
[INFO] 2019-03-15 22:54:42 [cn.itcast.dashboard.Main] - DAU|4645|领取优惠券|2019-03-15 07:40:29
[INFO] 2019-03-15 22:54:44 [cn.itcast.dashboard.Main] - DAU|3482|领取优惠券|2019-03-15 18:34:04
[INFO] 2019-03-15 22:54:48 [cn.itcast.dashboard.Main] - DAU|5607|加入收藏|2019-03-15 22:44:09
[INFO] 2019-03-15 22:54:50 [cn.itcast.dashboard.Main] - DAU|9619|加入收藏|2019-03-15 21:39:47
[INFO] 2019-03-15 22:54:53 [cn.itcast.dashboard.Main] - DAU|7666|加入收藏|2019-03-15 17:47:18
[INFO] 2019-03-15 22:54:54 [cn.itcast.dashboard.Main] - DAU|4871|提交订单|2019-03-15 02:36:27
[INFO] 2019-03-15 22:54:55 [cn.itcast.dashboard.Main] - DAU|7126|加入收藏|2019-03-15 16:11:06
[INFO] 2019-03-15 22:55:00 [cn.itcast.dashboard.Main] - DAU|9606|评论商品|2019-03-15 02:12:00
[INFO] 2019-03-15 22:55:02 [cn.itcast.dashboard.Main] - DAU|7698|查看订单|2019-03-15 08:17:01
可以看到数据同步到ES中
Kibana启动及配置
启动 Kibana
./bin/kibana
添加Logstash索引
discover 界面可以看到数据
创建自定义图表条形图并保存
创建自定义图表 饼图并保存
创建自定义dashboard 并添加我们自定义的图表
模拟产生日志数据可以看到图表实时变化