随着系统节点越来越多, 看应用日志成为一个头疼的事情, 所以想做一个应用日志收集. 看了一下log4j的SocketAppender, 居然使用java 自带序列化, 还不提供server, 挺讨厌. 倒不如自己写一个.
系统需求如下:
- 聚合多台服务器日志
- 无过大性能损耗
- 近实时
- 提供log4j和erlang客户端
简单用erlang实现了一个
server
地址 : https://github.com/haitaoyao/elogserver
使用方法:
rebar comile && cd rel && rebar generate
生成elogserver文件夹即server端. 启动时会读取/data/config/elogserver/elogserver.config, 示例配置文件如下:
{data_path, ["/tmp/elogserver/"]}.
配置日志存储路径.
${elogserver_home}/bin/elogserver start 后,系统监听9512端口
log4j client
地址: https://github.com/haitaoyao/elogserver-log4j
配置:
<appender name="elogserverAppender" class="elogserver.log4j.ELogServerAppender">
<param name = "category" value = "test_log" />
<param name = "logFileName" value = "test.log" />
<param name = "logServerAddress" value = "dctest:9512" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{yyyy-MM-dd HH:mm:ss} %p %t %c - %m%n" />
</layout>
</appender>
其中, category为系统名称, 例如 hadoop. logFileName为最终日志文件名称, 例如datanode.log, logServerAddress为server端地址
需要注意的是, 这个appender是同步发送日志到server, 如果发现性能问题可以考虑配合AsyncAppender使用.
erlang client
地址: https://github.com/haitaoyao/elogclient
erlang客户端其实是lager的一个backend实现. 需使用lager
最终日志样例:
10.20.100.34:49588 2012-09-17 09:57:09 ClientCnxn [INFO] Opening socket connection to server DC-hadoop08/10.20.99.223:2181
10.20.100.34:49588 2012-09-17 09:57:09 ClientCnxn [INFO] Socket connection established to DC-hadoop08/10.20.99.223:2181, initiating session
10.20.100.34:49588 2012-09-17 09:57:13 ClientCnxn [INFO] Client session timed out, have not heard from server in 4000ms for sessionid 0x8391f7a7f900040, closing socket connection and attempting reconnect
10.20.100.34:49588 2012-09-17 09:57:14 ClientCnxn [INFO] Opening socket connection to server DC-hadoop02/10.20.99.217:2181
10.20.100.34:49588 2012-09-17 09:57:14 ClientCnxn [INFO] Socket connection established to DC-hadoop02/10.20.99.217:2181, initiating session
10.20.100.34:49588 2012-09-17 09:57:14 ClientCnxn [INFO] Session establishment complete on server DC-hadoop02/10.20.99.217:2181, sessionid = 0x8391f7a7f900040, negotiated timeout = 20000
10.20.100.34:49588 2012-09-17 09:57:14 ConnectionStateManager [INFO] State change: RECONNECTED
10.20.100.34:49588 2012-09-17 09:57:14 ConnectionStateManager [WARN] There are no ConnectionStateListeners registered.
每行开始为日志来源IP:PORT
将所有的应用日志收集到同一个文件后, 分析类似zookeeper的问题要方便的多.
--EOF--