syslog-ng配置
syslog-ng的配置非常简单直观,于是乎配置好看看怎么用它实时收集日志。有两台服务器,一台服务器盯着error_log文件,一旦发现其有新数据,立即将日志发到另一台收集服务器。
client服务器配置如下。
123456789#将文件作为src
source s_blender_error_log {
file("/home/admin/kingso/logs/error_log_blender");
};
#收集到的日志发送给172.25.61.92
destination d_blender_error_log { tcp("172.25.61.92" port(514)); };
log { source(s_blender_error_log); destination(d_blender_error_log);};
server服务器配置如下。
0102030405060708091011#监听514端口
source s_kingso {
tcp(ip(0.0.0.0) port(514));
};
#设置输入文件及其相关的属性
destination d_kingso {
file("/home/admin/error.log" owner(admin) group(admin) perm(0755));
};
log { source(s_kingso); destination(d_kingso); };
最后收集到的日志如下。
1Mar 13 20:32:56 s061090.cm5 moximoximoximoximoxi
syslog-ng有一个很有意思的功能,那就是在destination里面支持program,可以对符合条件的日志支持操作,比如报警等等。下面这个配置会将所有包含error的日志输出到一个指定的文件中。为了提高性能,建议这个程序从标准输入读入数据,如果没有数据就阻塞等待,不要轻易退出。
123destination d_kingso {
program("grep error >> /tmp/eerroorr.log");
};
filter支持内容过滤,比如要把服务器上所有core信息都收集到一台服务器上,只要判断kern信息中是否有segfault就可以,配置如下。
1234filter f_kingso_core {
facility(kern) and
match("segfault" value("MESSAGE"));
}
如果要用syslog-ng收集大集群的日志,需要设置下max-connections这个参数,它的默认值是10,实在是太小了。如果不改这个配置,netstat会出现syslog-ng导致的大量TIME_WAIT。将其放大之后,可以算一下ESTABLISHED的链接数是不是和集群服务器数量相同。
syslog-ng服务器默认情况下收集到的日志是:”Apr 1 17:42:33 hostname blabla”。月份后面可能有一个空格(大于10号)也可能有两个空格(小于10号),这样对消息的切分就会有问题,这个时候必须通过template来规范消息的格式。syslog-ng支持定义一个template模版,以后的template都可以使用改模版,非常方便。下面这个配置就是program和file两个destination共用一个t_kingso模版。
123456template t_kingso { template("$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC\t$FULLHOST\t$MSGHDR$MSG\n"); };
destination d_kingso_blender {
program("exec python -u /home/henshao/log_monitor/log_monitor.py -c /home/henshao/log_monitor/log_monitor.conf -m blender >> /tmp/log_monitor.log 2>&1" template(t_kingso) );
file( "/home/admin/logs/blender.log.$S_YEAR-$S_MONTH-$S_DAY" create_dirs(yes) owner(admin) group(admin) perm(0755) template(t_kingso) );
};
我发现RHEL5上默认安装了syslogd、syslog-ng和rsyslog。syslogd历史最为悠久,wikipedia上的资料上讲它当初是sendmail项目的一部分,因为非常有用,所以逐渐成为所有unix-like系统的标准组件。syslog-ng则是syslogd的升级版本,是一个公司开发的,除了开源免费版还有收费的高级版,有一篇不错的文章: 专访syslog-ng 2.0开发人员Balazs Scheidler。rsyslog则最为年轻,作者想做一个log系统和syslog-ng竞争。
推荐一些不错的资料。
2、syslog-ng v2.0 reference manual
Syslog-ng安装及配置(2010-04-26 13:37:46)
注:当前使用版本为syslog-ng-2.0.9//使用以下RPM或是使用eventlog的源码包rpm -ivh eventlog-0.2.5-1.el5.i386.rpmrpm -ivh eventlog-devel-0.2.5-1.el4.i386.rpm ./configure --prefix=/usr/local/syslog-ng --sysconfdir=/etc make && make install mkfifo /tmp/mysql.pipe//创建mysql管道,接收到的日志写入该管道,通过期写入Mysql chkconfig --add syslog-ng chkconfig syslog-ng on chkconfig --add sqlsyslogd chkconfig sqlsyslogd on web界面采用php-syslog-ng来查看日志
以下是所用到的配置文件及服务文件
Syslog-ng.conf配置文件/etc/syslog-ng.conf ## configuration file for syslog-ng, customized for remote logging# source s_internal { internal(); };destination d_syslognglog { file("/var/log/syslog-ng.log"); };log { source(s_internal); destination(d_syslognglog); }; # Local sources, filters and destinations are commented out# If you want to replace sysklogd simply uncomment the following# parts and disable sysklogd## Local sources##source s_local {#unix-dgram("/dev/log"); #file("/proc/kmsg" log_prefix "kernel:"); #};## Local filters##filter f_messages { level(info..emerg); };#filter f_secure { facility(authpriv); };#filter f_mail { facility(mail); };#filter f_cron { facility(cron); };#filter f_emerg { level(emerg); };#filter f_spooler { level(crit..emerg) and facility(uucp, news); };#filter f_local7 { facility(local7); };## Local destinations##destination d_messages { file("/var/log/messages"); };#destination d_secure { file("/var/log/secure"); };#destination d_maillog { file("/var/log/maillog"); };#destination d_cron { file("/var/log/cron"); };#destination d_console { usertty("root"); };#destination d_spooler { file("/var/log/spooler"); };#destination d_bootlog { file("/var/log/boot.log"); };## Local logs - order DOES matter !##log { source(s_local); filter(f_emerg); destination(d_console); };#log { source(s_local); filter(f_secure); destination(d_secure); flags(final); };#log { source(s_local); filter(f_maillog); destination(d_maillog); flags(final); };#log { source(s_local); filter(f_cron); destination(d_cron); flags(final); };#log { source(s_local); filter(f_spooler); destination(d_spooler); };#log { source(s_local); filter(f_local7); destination(d_bootlog); };#log { source(s_local); filter(f_messages); destination(d_messages); }; # Remote loggingsource s_remote {tcp(ip(0.0.0.0) port(514)); udp(ip(0.0.0.0) port(514)); }; options { chain_hostnames(no); create_dirs (no); dir_perm(0755); dns_cache(yes); keep_hostname(yes); log_fifo_size(2048); log_msg_size(8192); long_hostnames(on); perm(0644); stats(3600); sync(0); time_reopen (10); use_dns(yes); use_fqdn(yes); }; destination d_separatedbyhosts { pipe("/tmp/mysql.pipe" template("INSERT INTO logs(host, facility, priority, level, tag, datetime, program, msg) VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC','$PROGRAM', '$MSG' );\n") template-escape(yes)); }; log { source(s_remote); destination(d_separatedbyhosts); }; #---------------------------------------------------------------------- # Sources #---------------------------------------------------------------------- # For Linux #---------------------------------------------------------------------- source s_stream { unix-stream("/dev/log"); }; source s_internal { internal(); }; source s_kernel { pipe("/proc/kmsg" log_prefix("kernel: ")); }; source s_tcp { tcp(port(4800) keep-alive(yes) max_connections(100)); }; #---------------------------------------------------------------------- # Piping method #---------------------------------------------------------------------- destination database { pipe("/tmp/mysql.pipe" template("INSERT INTO logs(host, facility, priority, level, tag, datetime, program, msg) VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC','$PROGRAM', '$MSG' );\n") template-escape(yes)); }; #---------------------------------------------------------------------- # Logging to a database #---------------------------------------------------------------------- log { source(s_stream); source(s_internal); source(s_kernel); destination(database); };
syslog-ng服务文件/etc/init.d/syslog-ng ############################################################### # # Program: syslog-ng init script for Red Hat # ############################################################### # the following information is for use by chkconfig # if you are want to manage this through chkconfig (as you should), you must # first must add syslog-ng to chkconfig's list of startup scripts it # manages by typing: # # chkconfig --add syslog-ng # # DO NOT CHANGE THESE LINES (unless you know what you are doing) # chkconfig: 2345 12 88 # description: syslog-ng is the next generation of the syslog daemon. \ # syslog-ng gives you the flexibility of logging not only by facility and \ # severity, but also by host, message content, date, etc. it can also replace \ # klogd's function of logging kernel messages # # This following block of lines is correct, do not change! (for more info, see # http://www.linuxbase.org/spec/refspecs/...facilname.html) ### BEGIN INIT INFO # Provides: $syslog ### END INIT INFO ################################################################ # This is an init script for syslog-ng on the Linux platform. # # It totally relies on the Redhat function library and works the same # way as other typical Redhat init scripts. # # # Platforms (tested): Linux (Redhat 7.3) # # # Author: Gregor Binder <gbinder@sysfive.com> # Changed: October 10, 2000 # # Last Changed: September 27, 2002 # Updated by: Diane Davidowicz # changes: Brought the start script up to snuff as far as compliance # with managing the startup script through chkconfig; # added PATH variable ability to hook in path to syslog-ng (if # its necessary); converted init script format to the # standard init script format in Red Hat (7.3 to be exact) # including using the /etc/sysconfig/syslog-ng file to # managed the arguments to syslog-ng without changing this # script, and disabled klogd but noted where and under what # conditions it should be enabled. HAPPY LOGGING. # # Copyright © 2000 by sysfive.com GmbH, All rights reserved. # # ############################################################### # # configuration # INIT_PROG="/usr/local/syslog-ng/sbin/syslog-ng" # Full path to daemon INIT_OPTS="" # options passed to daemon # # Source Redhat function library. # . /etc/rc.d/init.d/functions # Tack on path to syslog-ng if not already in PATH SYSLOGNG_PATH=":/usr/local/syslog-ng/sbin" PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/syslog-ng/sbinINIT_NAME=`basename "$INIT_PROG"` # /etc/sysconfig/ is the standard way to pull in options for a daemon to use. # Source config if [ -f /etc/sysconfig/syslog-ng ] ; then . /etc/sysconfig/syslog-ng else SYSLOGNG_OPTIONS= fi RETVAL=0 umask 077 ulimit -c 0 # See how we were called. start() { echo -n "Starting $INIT_PROG: " #daemon $INIT_PROG $SYSLOGNG_OPTIONS daemon --check $INIT_PROG "$INIT_PROG $INIT_OPTS" RETVAL=$? echo [ $RETVAL -eq 0 ] && touch "/var/lock/subsys/${INIT_NAME}" return $RETVAL } stop() { echo -n "Stopping $INIT_PROG: " killproc $INIT_PROG RETVAL=$? echo [ $RETVAL -eq 0 ] && rm -f "/var/lock/subsys/${INIT_NAME}" return $RETVAL } rhstatus() { status $INIT_PROG } restart() { stop start } case "$1" in start) start ;; stop) stop ;; status) rhstatus ;; restart|reload) restart ;; condrestart) [ -f /var/lock/subsys/syslog-ng ] && restart || : ;; *) echo $"Usage: $0 {start|stop|status|restart|reload}" exit 1 esac exit $?
实时把日志导入数据库的服务文件/etc/init.d/sqlsyslogd #!/bin/bash # # sqlsyslogd This is a daemon that takes syslog-ng input and pipe it into # a MySQL database. # # chkconfig: 2345 98 10 # description: sqlsyslogd bridges syslog-ng and mysql. # author: Josh Kuo Thu 2004/08/12 13:21:56 PDT . /etc/rc.d/init.d/functions case "$1" in start) if [ -x /tmp/mysql.pipe ]; then mkfifo /tmp/mysql.pipe else # if the service is already running, do not start another one PIDS=`pidofproc mysql` if [ "$PIDS" ]; then echo "sqlsyslogd is already running." exit 1 fi mysql -u 用户名 -h localhost –p密码 数据名 < /tmp/mysql.pipe & #If you need to collect apache logs into mysql, uncomment the following two lines. #tail -f /usr/local/apache2/logs/access_log | logger -p info -t apache & #tail -f /usr/local/apache2/logs/error_log | logger -p notice -t apache & fi ;; stop ) killproc mysql #If you need to collect apache logs into mysql, uncomment the next line. #killproc tail ;; *) echo "Usage: sqlsyslogd {start|stop}" exit 1; esac exit 0;
syslog-ng
最新推荐文章于 2024-08-05 17:34:36 发布