syslog-ng

syslog-ng配置

syslog-ng的配置非常简单直观,于是乎配置好看看怎么用它实时收集日志。有两台服务器,一台服务器盯着error_log文件,一旦发现其有新数据,立即将日志发到另一台收集服务器。

client服务器配置如下。

?
1
2
3
4
5
6
7
8
9
#将文件作为src
source s_blender_error_log {
        file("/home/admin/kingso/logs/error_log_blender");
};
#收集到的日志发送给172.25.61.92
destination d_blender_error_log { tcp("172.25.61.92" port(514)); };
log { source(s_blender_error_log); destination(d_blender_error_log);};

server服务器配置如下。

?
01
02
03
04
05
06
07
08
09
10
11
#监听514端口
source s_kingso {
        tcp(ip(0.0.0.0) port(514));
};
#设置输入文件及其相关的属性
destination d_kingso {
        file("/home/admin/error.log" owner(admin) group(admin) perm(0755));
};
log { source(s_kingso); destination(d_kingso); };

最后收集到的日志如下。

?
1
Mar 13 20:32:56 s061090.cm5 moximoximoximoximoxi

syslog-ng有一个很有意思的功能,那就是在destination里面支持program,可以对符合条件的日志支持操作,比如报警等等。下面这个配置会将所有包含error的日志输出到一个指定的文件中。为了提高性能,建议这个程序从标准输入读入数据,如果没有数据就阻塞等待,不要轻易退出。

?
1
2
3
destination d_kingso {
        program("grep error >> /tmp/eerroorr.log");
};

filter支持内容过滤,比如要把服务器上所有core信息都收集到一台服务器上,只要判断kern信息中是否有segfault就可以,配置如下。

?
1
2
3
4
filter f_kingso_core {
    facility(kern) and
    match("segfault" value("MESSAGE"));
}

如果要用syslog-ng收集大集群的日志,需要设置下max-connections这个参数,它的默认值是10,实在是太小了。如果不改这个配置,netstat会出现syslog-ng导致的大量TIME_WAIT。将其放大之后,可以算一下ESTABLISHED的链接数是不是和集群服务器数量相同。

syslog-ng服务器默认情况下收集到的日志是:”Apr 1 17:42:33 hostname blabla”。月份后面可能有一个空格(大于10号)也可能有两个空格(小于10号),这样对消息的切分就会有问题,这个时候必须通过template来规范消息的格式。syslog-ng支持定义一个template模版,以后的template都可以使用改模版,非常方便。下面这个配置就是program和file两个destination共用一个t_kingso模版。

?
1
2
3
4
5
6
template t_kingso { template("$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC\t$FULLHOST\t$MSGHDR$MSG\n"); };
destination d_kingso_blender {
    program("exec python -u /home/henshao/log_monitor/log_monitor.py -c /home/henshao/log_monitor/log_monitor.conf -m blender >> /tmp/log_monitor.log 2>&1" template(t_kingso) );
    file( "/home/admin/logs/blender.log.$S_YEAR-$S_MONTH-$S_DAY" create_dirs(yes) owner(admin) group(admin) perm(0755) template(t_kingso) );
};

我发现RHEL5上默认安装了syslogd、syslog-ng和rsyslog。syslogd历史最为悠久,wikipedia上的资料上讲它当初是sendmail项目的一部分,因为非常有用,所以逐渐成为所有unix-like系统的标准组件。syslog-ng则是syslogd的升级版本,是一个公司开发的,除了开源免费版还有收费的高级版,有一篇不错的文章: 专访syslog-ng 2.0开发人员Balazs Scheidler。rsyslog则最为年轻,作者想做一个log系统和syslog-ng竞争。

推荐一些不错的资料。

1、Syslog-ng

2、syslog-ng v2.0 reference manual

 

 

 

Syslog-ng安装及配置(2010-04-26 13:37:46)

标签:

杂谈

分类: linux

注:当前使用版本为syslog-ng-2.0.9
//使用以下RPM或是使用eventlog的源码包
rpm -ivh eventlog-0.2.5-1.el5.i386.rpm
rpm -ivh eventlog-devel-0.2.5-1.el4.i386.rpm
./configure --prefix=/usr/local/syslog-ng --sysconfdir=/etc
make && make install
mkfifo /tmp/mysql.pipe  //创建mysql管道,接收到的日志写入该管道,通过期写入Mysql
chkconfig --add syslog-ng
chkconfig syslog-ng on
chkconfig --add sqlsyslogd
chkconfig sqlsyslogd on
web界面采用php-syslog-ng来查看日志

以下是所用到的配置文件及服务文件

Syslog-ng.conf配置文件
/etc/syslog-ng.conf
#
# configuration file for syslog-ng, customized for remote logging
#
source s_internal { internal(); };
destination d_syslognglog { file("/var/log/syslog-ng.log"); };
log { source(s_internal); destination(d_syslognglog); };
# Local sources, filters and destinations are commented out
# If you want to replace sysklogd simply uncomment the following
# parts and disable sysklogd
#
# Local sources
#
#source s_local {
    unix-dgram("/dev/log");
    file("/proc/kmsg" log_prefix "kernel:");
#};
#
# Local filters
#
#filter f_messages { level(info..emerg); };
#filter f_secure { facility(authpriv); };
#filter f_mail { facility(mail); };
#filter f_cron { facility(cron); };
#filter f_emerg { level(emerg); };
#filter f_spooler { level(crit..emerg) and facility(uucp, news); };
#filter f_local7 { facility(local7); };
#
# Local destinations
#
#destination d_messages { file("/var/log/messages"); };
#destination d_secure { file("/var/log/secure"); };
#destination d_maillog { file("/var/log/maillog"); };
#destination d_cron { file("/var/log/cron"); };
#destination d_console { usertty("root"); };
#destination d_spooler { file("/var/log/spooler"); };
#destination d_bootlog { file("/var/log/boot.log"); };
#
# Local logs - order DOES matter !
#
#log { source(s_local); filter(f_emerg); destination(d_console); };
#log { source(s_local); filter(f_secure); destination(d_secure); flags(final); };
#log { source(s_local); filter(f_maillog); destination(d_maillog); flags(final); };
#log { source(s_local); filter(f_cron); destination(d_cron); flags(final); };
#log { source(s_local); filter(f_spooler); destination(d_spooler); };
#log { source(s_local); filter(f_local7); destination(d_bootlog); };
#log { source(s_local); filter(f_messages); destination(d_messages); };
# Remote logging
source s_remote {
      tcp(ip(0.0.0.0) port(514));
      udp(ip(0.0.0.0) port(514));
};
options
{
chain_hostnames(no);
create_dirs (no);
dir_perm(0755);
dns_cache(yes);
keep_hostname(yes);
log_fifo_size(2048);
log_msg_size(8192);
long_hostnames(on);
perm(0644);
stats(3600);
sync(0);
time_reopen (10);
use_dns(yes);
use_fqdn(yes);
};
destination d_separatedbyhosts { pipe("/tmp/mysql.pipe" template("INSERT INTO logs
    (host, facility, priority, level, tag, datetime, program, msg)
    VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC','$PROGRAM', '$MSG' );\n") template-escape(yes)); };
log { source(s_remote); destination(d_separatedbyhosts); };
#----------------------------------------------------------------------
# Sources
#----------------------------------------------------------------------
# For Linux
#----------------------------------------------------------------------
source s_stream
{ unix-stream("/dev/log"); };
source s_internal
{ internal(); };
source s_kernel
{ pipe("/proc/kmsg" log_prefix("kernel: ")); };
source s_tcp
{ tcp(port(4800) keep-alive(yes) max_connections(100)); };
#----------------------------------------------------------------------
# Piping method
#----------------------------------------------------------------------
destination database { pipe("/tmp/mysql.pipe" template("INSERT INTO logs
    (host, facility, priority, level, tag, datetime, program, msg) VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC','$PROGRAM', '$MSG' );\n") template-escape(yes)); };
#----------------------------------------------------------------------
# Logging to a database
#----------------------------------------------------------------------
log { source(s_stream);
source(s_internal);
source(s_kernel); destination(database); };


syslog-ng服务文件
/etc/init.d/syslog-ng
###############################################################
#
# Program: syslog-ng init script for Red Hat
#
###############################################################
# the following information is for use by chkconfig
# if you are want to manage this through chkconfig (as you should), you must
# first must add syslog-ng to chkconfig's list of startup scripts it
# manages by typing:
#
# chkconfig --add syslog-ng
#
# DO NOT CHANGE THESE LINES (unless you know what you are doing)
# chkconfig: 2345 12 88
# description: syslog-ng is the next generation of the syslog daemon. \
# syslog-ng gives you the flexibility of logging not only by facility and \
# severity, but also by host, message content, date, etc. it can also replace \
# klogd's function of logging kernel messages
#
# This following block of lines is correct, do not change! (for more info, see
# http://www.linuxbase.org/spec/refspecs/...facilname.html)
### BEGIN INIT INFO
# Provides: $syslog
### END INIT INFO
###############################################################
#
# This is an init script for syslog-ng on the Linux platform.
#
# It totally relies on the Redhat function library and works the same
# way as other typical Redhat init scripts.
#
#
# Platforms (tested): Linux (Redhat 7.3)
#
#
# Author: Gregor Binder <gbinder@sysfive.com>
# Changed: October 10, 2000
#
# Last Changed: September 27, 2002
# Updated by: Diane Davidowicz
# changes: Brought the start script up to snuff as far as compliance
# with managing the startup script through chkconfig;
# added PATH variable ability to hook in path to syslog-ng (if
# its necessary); converted init script format to the
# standard init script format in Red Hat (7.3 to be exact)
# including using the /etc/sysconfig/syslog-ng file to
# managed the arguments to syslog-ng without changing this
# script, and disabled klogd but noted where and under what
# conditions it should be enabled. HAPPY LOGGING.
#
# Copyright &copy; 2000 by sysfive.com GmbH, All rights reserved.
#
#
###############################################################
#
# configuration
#
INIT_PROG="/usr/local/syslog-ng/sbin/syslog-ng" # Full path to daemon
INIT_OPTS="" # options passed to daemon
#
# Source Redhat function library.
#
. /etc/rc.d/init.d/functions
# Tack on path to syslog-ng if not already in PATH
SYSLOGNG_PATH=":/usr/local/syslog-ng/sbin"
PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/syslog-ng/sbin
INIT_NAME=`basename "$INIT_PROG"`
# /etc/sysconfig/ is the standard way to pull in options for a daemon to use.
# Source config
if [ -f /etc/sysconfig/syslog-ng ] ; then
. /etc/sysconfig/syslog-ng
else
SYSLOGNG_OPTIONS=
fi
RETVAL=0
umask 077
ulimit -c 0
# See how we were called.
start() {
echo -n "Starting $INIT_PROG: "
#daemon $INIT_PROG $SYSLOGNG_OPTIONS
daemon --check $INIT_PROG "$INIT_PROG $INIT_OPTS"
RETVAL=$?
echo
[ $RETVAL -eq 0 ] && touch "/var/lock/subsys/${INIT_NAME}"
return $RETVAL
}
stop() {
echo -n "Stopping $INIT_PROG: "
killproc $INIT_PROG
RETVAL=$?
echo
[ $RETVAL -eq 0 ] && rm -f "/var/lock/subsys/${INIT_NAME}"
return $RETVAL
}
rhstatus() {
status $INIT_PROG
}
restart() {
stop
start
}
case "$1" in
start)
start
;;
stop)
stop
;;
status)
rhstatus
;;
restart|reload)
restart
;;
condrestart)
[ -f /var/lock/subsys/syslog-ng ] && restart || :
;;
*)
echo $"Usage: $0 {start|stop|status|restart|reload}"
exit 1
esac
exit $?


实时把日志导入数据库的服务文件
/etc/init.d/sqlsyslogd
#!/bin/bash
#
# sqlsyslogd This is a daemon that takes syslog-ng input and pipe it into
# a MySQL database.
#
# chkconfig: 2345 98 10
# description: sqlsyslogd bridges syslog-ng and mysql.
# author: Josh Kuo Thu 2004/08/12 13:21:56 PDT
. /etc/rc.d/init.d/functions
case "$1" in
start)
if [ -x /tmp/mysql.pipe ]; then
mkfifo /tmp/mysql.pipe
else
# if the service is already running, do not start another one
PIDS=`pidofproc mysql`
if [ "$PIDS" ]; then
echo "sqlsyslogd is already running."
exit 1
fi
mysql -u 用户名 -h localhost –p密码 数据名 < /tmp/mysql.pipe &
#If you need to collect apache logs into mysql, uncomment the following two lines.
#tail -f /usr/local/apache2/logs/access_log | logger -p info -t apache &
#tail -f /usr/local/apache2/logs/error_log | logger -p notice -t apache &
fi
;;
stop )
killproc mysql
#If you need to collect apache logs into mysql, uncomment the next line.
#killproc tail
;;
*)
echo "Usage: sqlsyslogd {start|stop}"
exit 1;
esac
exit 0;

 

 

  • 0
    点赞
  • 1
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值