一、前言
公司有一项目出错,过了两个小时才发现它,为了第一时间发现错误,我们运维团队选择filebeat+logstash采集错误日志,zabbix实现报警。
二、配置流程
1.配置zabbix
#创建应用集
#在应用集里创建监控项
#创建触发器
2、配置filebeat
编辑filebeat配置文件
vi /etc/filebeat/filebeat.yml
filebeat.prospectors: - input_type: log paths: - /opt/ops/logs/easyopen.log #匹配时间格式开头的日志合并多行为一行(2019-06-14) multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}' multiline.negate: true multiline.match: after #过滤包含‘ERROR’字符串的行 include_lines: ['ERROR'] #添加logtype字段,在logstash 做条件判断 fields: logtype: easyopen_log #输出到logstah output.logstash: # The Logstash hosts Logstash 采用tcp采集模式 hosts: ["172.17.0.15:5044"]
![](https://i-blog.csdnimg.cn/blog_migrate/8f900a89c6347c561fdf2122f13be562.gif)
![](https://i-blog.csdnimg.cn/blog_migrate/961ddebeb323a10fe0623af514929fc1.gif)
2019-06-14 17:19:13.317 ERROR 13864 --- [ntLoopGroup-3-2] c.g.a.netty.ApiConfigServerHandler : 服务端出错 java.io.IOException: Connection reset by peer at sun.nio.ch.FileDispatcherImpl.read0(Native Method) ~[na:1.8.0_131] at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) ~[na:1.8.0_131] at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) ~[na:1.8.0_131] at sun.nio.ch.IOUtil.read(IOUtil.java:192) ~[na:1.8.0_131] at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380) ~[na:1.8.0_131] at io.netty.buffer.PooledUnsafeDirectByteBuf.setBytes(PooledUnsafeDirectByteBuf.java:288) ~[netty-all-4.1.25.Final.jar!/:4.1.25.Final] at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1108) ~[netty-all-4.1.25.Final.jar!/:4.1.25.Final] at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:345) ~[netty-all-4.1.25.Final.jar!/:4.1.25.Final] at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:148) ~[netty-all-4.1.25.Final.jar!/:4.1.25.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:647) [netty-all-4.1.25.Final.jar!/:4.1.25.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:582) [netty-all-4.1.25.Final.jar!/:4.1.25.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:499) [netty-all-4.1.25.Final.jar!/:4.1.25.Final] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:461) [netty-all-4.1.25.Final.jar!/:4.1.25.Final] at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884) [netty-all-4.1.25.Final.jar!/:4.1.25.Final] at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-all-4.1.25.Final.jar!/:4.1.25.Final] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_131]
#重启filebeat服务
systemctl restart filebeat.service
3.配置logstash
[root@Ops-system conf.d]# cat FK-middleware.conf
input { beats {port => 5044} } filter { #logtype 这个字段用于做条件判断是哪个服务的日志 if [fields][logtype] == "easyopen_log" { mutate { add_field => [ "[@metadata][zabbix_key]" , "easyopen" ] #zabbix服务端定义的键名 add_field => [ "[@metadata][zabbix_host]" , "FK-tomcat" ] #zabbix客户端定义的主机名 } } } output { stdout{ codec=>rubydebug } zabbix { zabbix_host => "[@metadata][zabbix_host]" zabbix_key => "[@metadata][zabbix_key]" zabbix_server_host => "127.0.0.1" #zabbix服务端地址 zabbix_server_port => "10051" #zabbix服务端端口 zabbix_value => "message" } }
#重启logstash
/etc/init.d/logstash restart
4.展示效果
邮件报警显示