目的:我们使用filebeat来接受日志,并将日志做一个简单的处理,然后发给logstash,使用logstash的filter的grok来匹配日志,将日志解析成json格式并将日志输出在logstash的控制台上.
首先看一下我们的日志样例,这是一个标准的java的log4j日志:
2019-02-28 10:24:48 org.tinyradius.util.RadiusClient.authenticate(RadiusClient.java:81) INFO (org.tinyradius.util.RadiusClient:81)- send Access-Request packet: Access-Request, ID 7
User-Name: admin
2019-02-28 10:24:48 org.tinyradius.util.RadiusClient.authenticate(RadiusClient.java:85) INFO (org.tinyradius.util.RadiusClient:85)- received packet: Access-Accept, ID 7
Login-LAT-Group: 0x313233343536
2019-02-28 10:24:49 com.bas.pub.util.PropertiesHelper.getPropFile(PropertiesHelper.java:44) INFO (com.bas.pub.util.PropertiesHelper:44)- 1-propFile==config.properties
2019-02-28 10:24:49 com.bas.pub.util.PropertiesHelper.getPropFile(PropertiesHelper.java:49) INFO (com.bas.pub.util.PropertiesHelper:49)- 3-in==java.io.ByteArrayInputStream@32250194
2019-02-28 15:06:24 org.tinyradius.util.RadiusClient.authenticate(RadiusClient.java:81) INFO (org.tinyradius.util.RadiusClient:81)- send Access-Request packet: Access-Request, ID 8
User-Name: admin
2019-02-28 15:06:24 org.tinyradius.util.RadiusClient.communicate(RadiusClient.java:262) INFO (org.tinyradius.util.RadiusClient:262)- communication failure, retry 1
2019-02-28 15:06:24 org.tinyradius.util.RadiusClient.communicate(RadiusClient.java:262) INFO (org.tinyradius.util.RadiusClient:262)- communication failure, retry 2
2019-02-28 15:06:24 org.tinyradius.util.RadiusClient.communicate(RadiusClient.java:262) INFO (org.tinyradius.util.RadiusClient:262)- communication failure, retry 3
2019-02-28 15:06:24 org.tinyradius.util.RadiusClient.communicate(RadiusClient.java:262) INFO (org.tinyradius.util.RadiusClient:262)- communication failure, retry 4
2019-02-28 15:06:24 org.tinyradius.util.RadiusClient.communicate(RadiusClient.java:262) INFO (org.tinyradius.util.RadiusClient:262)- communication failure, retry 5
2019-02-28 15:06:24 org.tinyradius.util.RadiusClient.communicate(RadiusClient.java:255) ERROR (org.tinyradius.util.RadiusClient:255)- communication failure (timeout), no more retries
2019-02-28 15:06:24 com.erp.action.LoginAction.load(LoginAction.java:230) ERROR (com.erp.action.LoginAction:230)- Radius认证异常,请与系统管理员联系
java.net.SocketTimeoutException: Receive timed out
at java.net.DualStackPlainDatagramSocketImpl.socketReceiveOrPeekData(Native Method)
at java.net.DualStackPlainDatagramSocketImpl.receive0(DualStackPlainDatagramSocketImpl.java:121)
at java.net.AbstractPlainDatagramSocketImpl.receive(AbstractPlainDatagramSocketImpl.java:146)
at java.net.DatagramSocket.receive(DatagramSocket.java:816)
at org.tinyradius.util.RadiusClient.communicate(RadiusClient.java:249)
at org.tinyradius.util.RadiusClient.authenticate(RadiusClient.java:83)
at org.tinyradius.util.RadiusClient.authenticate(RadiusClient.java:65)
at com.erp.action.LoginAction.load(LoginAction.java:207)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at ognl.OgnlRuntime.invokeMethod(OgnlRuntime.java:892)
at ognl.OgnlRuntime.callAppropriateMethod(OgnlRuntime.java:1294)
at ognl.ObjectMethodAccessor.callMethod(ObjectMethodAccessor.java:68)
at com.opensymphony.xwork2.ognl.accessor.XWorkMethodAccessor.callMethodWithDebugInfo(XWorkMethodAccessor.java:117)
at com.opensymphony.xwork2.ognl.accessor.XWorkMethodAccessor.callMethod(XWorkMethodAccessor.java:108)
at ognl.OgnlRuntime.callMethod(OgnlRuntime.java:1370)
at ognl.ASTMethod.getValueBody(ASTMethod.java:91)
at ognl.SimpleNode.evaluateGetValueBody(SimpleNode.java:212)
at ognl.SimpleNode.getValue(SimpleNode.java:258)
at ognl.Ognl.getValue(Ognl.java:467)
at ognl.Ognl.getValue(Ognl.java:431)
at com.opensymphony.xwork2.ognl.OgnlUtil$3.execute(OgnlUtil.java:352)
at com.opensymphony.xwork2.ognl.OgnlUtil.compileAndExecuteMethod(OgnlUtil.java:404)
at com.opensymphony.xwork2.ognl.OgnlUtil.callMethod(OgnlUtil.java:350)
at com.opensymphony.xwork2.DefaultActionInvocation.invokeAction(DefaultActionInvocation.java:430)
at com.opensymphony.xwork2.DefaultActionInvocation.invokeActionOnly(DefaultActionInvo