rsyslog 是sysklogd的加强版替代品。并提供扩展的过滤,消息的加密保护中继,各种配置选项,输入和输出模块,支持通过TCP或UDP协议进行传输。 请注意,rsyslog与sysklogd兼容。
通过编辑/etc/rsyslog.conf、或者/etc/rsyslog.d/*.conf(因为/etc/rsyslog.conf中的$IncludeConfig /etc/rsyslog.d/*.conf而生效)等配置rsyslog的行为。在这里,您可以指定由过滤器和操作部件组成的全局指令,模块和规则。
默认/etc/rsyslog.conf配置文件
# rsyslog configuration file
# For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html
# If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html
#### MODULES ####
# The imjournal module bellow is now used as a message source instead of imuxsock.
$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
$ModLoad imjournal # provides access to the systemd journal
#$ModLoad imklog # reads kernel messages (the same are read from journald)
#$ModLoad immark # provides --MARK-- message capability
# Provides UDP syslog reception
#$ModLoad imudp
#$UDPServerRun 514
# Provides TCP syslog reception
#$ModLoad imtcp
#$InputTCPServerRun 514
#### GLOBAL DIRECTIVES ####
# Where to place auxiliary files
$WorkDirectory /var/lib/rsyslog
# Use default timestamp format
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
# File syncing capability is disabled by default. This feature is usually not required,
# not useful and an extreme performance hit
#$ActionFileEnableSync on
# Include all config files in /etc/rsyslog.d/
$IncludeConfig /etc/rsyslog.d/*.conf
# Turn off message reception via local log socket;
# local messages are retrieved through imjournal now.
$OmitLocalLogging on
# File to store the position in the journal
$IMJournalStateFile imjournal.state
#### RULES ####
# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.* /dev/console
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none /var/log/messages
# The authpriv file has restricted access.
authpriv.* /var/log/secure
# Log all the mail messages in one place.
mail.* -/var/log/maillog
# Log cron stuff
cron.* /var/log/cron
# Everybody gets emergency messages
*.emerg :omusrmsg:*
# Save news errors of level crit and higher in a special file.
uucp,news.crit /var/log/spooler
# Save boot messages also to boot.log
local7.* /var/log/boot.log
# ### begin forwarding rule ###
# The statement between the begin ... end define a SINGLE forwarding
# rule. They belong together, do NOT split them. If you create multiple
# forwarding rules, duplicate the whole block!
# Remote Logging (we use TCP for reliable delivery)
#
# An on-disk queue is created for this action. If the remote host is
# down, messages are spooled to disk and sent when it is up again.
#$ActionQueueFileName fwdRule1 # unique name prefix for spool files
#$ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible)
#$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
#$ActionQueueType LinkedList # run asynchronously
#$ActionResumeRetryCount -1 # infinite retries if host is down
# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
#*.* @@remote-host:514
# ### end of the forwarding rule ###
Filter
提供过滤syslog message的方式
可以基于Facility/优先级 Facility/属性 Facility/表达式
FACILITY指定产生特定系统日志消息的子系统。 例如,邮件子系统处理所有与邮件相关的系统日志消息。 FACILITY可以用以下关键字之一(或数字代码)表示:
kern(0),user(1),mail(2),daemon(3),auth(4),syslog(5),lpr( 6),新闻(7),cron(8),authpriv(9),ftp(10)和local0到local7(16-23)。
Facility/优先级
优先级PRIORITY指定系统日志消息的优先级。 可以通过以下关键字之一(或数字)来表示优先级: debug (7), info (6), notice (5), warning (4), err (3), crit (2), alert (1), and emerg (0).
FACILITY和PRIORITY不区分大小写。用.间隔
FACILITY.PRIORITY
FACILITY.PRIORITY 指定优先级及其以上的优先级
FACILITY.=PRIORITY 指定优先级
FACILITY.!PRIORITY 指定优先级除外
FACILITY.none 未给定优先级
FACILITY.* 任意优先级
同FACILITY的多个PRIORITY需要用逗号间隔 e.g. cron.!info,!debug
如果同一行指定多个FACILITY 需要用分号间隔
Facility/属性
属性PROPERTY可以按照属性筛选消息。用冒号:开头;
:PROPERTY, [!]COMPARE_OPERATION, "STRING"
COMPARE_OPERATION支持 contains, contains_i(不区分大小写), isequal, startswith,startswith(不区分大小写),regex(基本正则), ereregex(扩展正则), isempty(属性为空,则丢弃)
[!]逻辑运算符,可选的,目前只支持!非
STRING属性指定与该属性提供的文本进行比较的值。 该值必须用引号引起来。 要在字符串中转义某些字符(例如,引号(“)),请使用反斜杠字符(\)。
:msg, contains, "error"
:hostname, isequal, "host1"
:msg, !regex, "fatal .* error"
Facility/表达式
表达式EXPRESSION 根据算数,boolean,字符串运算筛选消息。复杂结构可以使用RainerScript(rsyslog自己的脚本语言)
if EXPRESSION then ACTION else ACTION
e.g.
if $programname == 'prog1' then {
action(type="omfile" file="/var/log/prog1.log")
if $msg contains 'test' then
action(type="omfile" file="/var/log/prog1test.log")
else
action(type="omfile" file="/var/log/prog1notest.log")
}
Actions
- 将log message保存到log文件
FILTER PATH
e.g.
cron.* /var/log/cron.log
FILTER -PATH
禁止磁盘的实时同步,会丢消息,但是性能好
FILTER ?DynamicFile
指定动态文件,动态文件由?和template组成。DynamicFile是修改输出路径的预定义模板的名称。也可以使用-禁止实时同步,如果有多个模板用分号间隔。
2. 通过网络发送log message
@(zNUMBER)HOST:PORT
@ 表示UDP 协议
@@ 表示TCP协议
可选的zNUMBER设置为系统日志消息启用zlib压缩。 NUMBER属性指定压缩级别(从1 –最低到9 –最大)。
将IPv6地址指定为host时,请将该地址括在方括号([,])中。
e.g.
3.输出到channel中
输出channel主要用于指定日志文件可以增长到的最大大小。然后执行相关action的一种方式。
$outchannel NAME, FILE_NAME, MAX_SIZE, ACTION
e.g.
定义channel
$outchannel log_rotation, /var/log/test_log.log, 104857600, /home/joe/log_rotation_script
用到规则中
cron.* :omfile:$log_rotation
实现脚本/home/joe/log_rotation_script
mv -f ${1} ${1}.1
4.发送给指定用户
5.指定一段程序
FILTER ^EXECUTABLE; TEMPLATE
6.保存到DB中(目前只支持mysql和postgreSQL)
需要引入想要的module
module(load="ommysql") # Output module for MySQL support
module(load="ompgsql") # Output module for PostgreSQL support
:PLUGIN:DB_HOST,DB_NAME,DB_USER,DB_PASSWORD;TEMPLATE
7.扔掉
使用stop关键字。rsyslog7中用~代替了stop关键字
cron.* stop
----
指定多个ACTION
FILTER ACTION
& ACTION
& ACTION
Templates
rsyslog生成的任何输出都可以使用模板根据您的需要进行修改和格式化。 创建模板:
template(name="TEMPLATE_NAME" type="string" string="text %PROPERTY% more text" [option.OPTION="on"])
name 必须唯一
type可以是list,subtree, string, plugin
string是实际的模板文本,在文本内可以使用特殊字符。\n换号,\r回车符。用%包裹property
生成动态文件名:
通过用 property作为文件名中的一部分。
e.g.
#template(name="clw" type="list") {
# constant(value="/var/log/clw/")
# property(name="timegenerated" dateFormat="rfc3339" position.from="1" position.to="16")
# constant(value="-test.log")
#}
template(name="clw" type="string" string="/var/log/clw/%timegenerated:1:10:date-rfc3339%-test.log")
ftp.* ?clw
Properties
在模板内定义的属性(在两个百分号(%)之间)允许通过使用属性替换器来访问syslog消息的各种内容。 要在模板内(两个引号(“ ..."之间)定义属性,请使用以下语法:
%PROPERTY_NAME:FROM_CHAR:TO_CHAR:OPTION%
PROPERTY_NAME: ref https://www.rsyslog.com/doc/master/configuration/properties.html
FROM_CHAR/TO_CHAR_OPTION: ref https://www.rsyslog.com/doc/master/configuration/property_replacer.html
一些尝试:
1.尝试动态文件名,其中用timegenerated 属性,可以实现日志文件按时间的自然分割;
跨文件的操作不会丢失日志信息。但是文件句柄不会主动关闭,看现象是最多支持10个文件句柄,如果超过,会关闭最早的文件句柄。
May 6 16:09:59 xx-xxx vsftpd[12608]: [user] OK DOWNLOAD: Client "::ffff:xxx.xxx.xxx.xxx", "/opt/repository/common/mchange-commons-java-0.2.11.jar", 606472 bytes, 97507.05Kbyte/sec
[root@xx-xxx clw]# grep DOWNLOAD ./2021-05-06T16:09-test.log|wc -l
82
[root@xx-xxx clw]# grep DOWNLOAD ./2021-05-06T16:10-test.log
May 6 16:10:00 xx-xxx vsftpd[12608]: [user] OK DOWNLOAD: Client "::ffff:xxx.xxx.xxx.xxx", "/opt/repository/common/mimepull-1.9.11.jar", 67125 bytes, 50039.51Kbyte/sec
[root@xx-xxx clw]# grep DOWNLOAD ./2021-05-06T16:10-test.log|wc -l
72
[user@G9_111 ~]$ ll *.jar | wc -l
154
ps -ef | grep rsyslog 获取pid
ls -la /proc/pid/fd
[root@xx-xxx clw]# ls -la /proc/8484/fd
total 0
dr-x------ 2 root root 0 May 6 15:50 .
dr-xr-xr-x 9 root root 0 May 6 15:50 ..
lr-x------ 1 root root 64 May 6 15:50 0 -> /dev/null
l-wx------ 1 root root 64 May 6 15:50 1 -> /dev/null
l-wx------ 1 root root 64 May 6 15:50 10 -> /var/log/clw/2021-05-06T16:23-test.log
l-wx------ 1 root root 64 May 6 15:50 11 -> /var/log/clw/2021-05-06T16:24-test.log
l-wx------ 1 root root 64 May 6 15:50 12 -> /var/log/cron
l-wx------ 1 root root 64 May 6 15:50 13 -> /var/log/clw/2021-05-06T16:25-test.log
l-wx------ 1 root root 64 May 6 15:50 14 -> /var/log/clw/2021-05-06T16:26-test.log
l-wx------ 1 root root 64 May 6 15:50 15 -> /var/log/clw/2021-05-06T16:43-test.log
l-wx------ 1 root root 64 May 6 15:50 16 -> /var/log/clw/2021-05-06T16:10-test.log
l-wx------ 1 root root 64 May 6 15:50 17 -> /var/log/clw/2021-05-06T16:18-test.log
l-wx------ 1 root root 64 May 6 15:50 18 -> /var/log/clw/2021-05-06T16:19-test.log
l-wx------ 1 root root 64 May 6 15:50 2 -> /dev/null
lr-x------ 1 root root 64 May 6 15:50 3 -> anon_inode:inotify
l-wx------ 1 root root 64 May 6 15:50 4 -> /var/log/messages
lr-x------ 1 root root 64 May 6 15:50 5 -> /run/log/journal/d0cad179a1b6e7a1f65eb084e2c2404d/system.journal
lrwx------ 1 root root 64 May 6 15:50 6 -> socket:[117881]
l-wx------ 1 root root 64 May 6 15:50 7 -> /var/log/secure
l-wx------ 1 root root 64 May 6 15:50 8 -> /var/log/clw/2021-05-06T16:21-test.log
l-wx------ 1 root root 64 May 6 15:50 9 -> /var/log/clw/2021-05-06T16:22-test.log
[root@xx-xxx clw]# ls -la /proc/8484/fd
total 0
dr-x------ 2 root root 0 May 6 15:50 .
dr-xr-xr-x 9 root root 0 May 6 15:50 ..
lr-x------ 1 root root 64 May 6 15:50 0 -> /dev/null
l-wx------ 1 root root 64 May 6 15:50 1 -> /dev/null
l-wx------ 1 root root 64 May 6 15:50 10 -> /var/log/clw/2021-05-06T16:23-test.log
l-wx------ 1 root root 64 May 6 15:50 11 -> /var/log/clw/2021-05-06T16:24-test.log
l-wx------ 1 root root 64 May 6 15:50 12 -> /var/log/cron
l-wx------ 1 root root 64 May 6 15:50 13 -> /var/log/clw/2021-05-06T16:25-test.log
l-wx------ 1 root root 64 May 6 15:50 14 -> /var/log/clw/2021-05-06T16:26-test.log
l-wx------ 1 root root 64 May 6 15:50 15 -> /var/log/clw/2021-05-06T16:43-test.log
l-wx------ 1 root root 64 May 6 15:50 16 -> /var/log/clw/2021-05-06T16:45-test.log
l-wx------ 1 root root 64 May 6 15:50 17 -> /var/log/clw/2021-05-06T16:18-test.log
l-wx------ 1 root root 64 May 6 15:50 18 -> /var/log/clw/2021-05-06T16:19-test.log
l-wx------ 1 root root 64 May 6 15:50 2 -> /dev/null
lr-x------ 1 root root 64 May 6 15:50 3 -> anon_inode:inotify
l-wx------ 1 root root 64 May 6 15:50 4 -> /var/log/messages
lr-x------ 1 root root 64 May 6 15:50 5 -> /run/log/journal/d0cad179a1b6e7a1f65eb084e2c2404d/system.journal
lrwx------ 1 root root 64 May 6 15:50 6 -> socket:[117881]
l-wx------ 1 root root 64 May 6 15:50 7 -> /var/log/secure
l-wx------ 1 root root 64 May 6 15:50 8 -> /var/log/clw/2021-05-06T16:21-test.log
l-wx------ 1 root root 64 May 6 15:50 9 -> /var/log/clw/2021-05-06T16:22-test.log