Syslog-ng+snoopy 日志审计解决方案
简介：本解决方案通过 syslog-ng+snoopy实现完善的LINUX/UNIX日志集中存储和用户操作行为记录审计功能。Snoopy可以实时捕获用户SHELL下输入的命令并传递给syslog，由于系统默认的syslog功能简单，不支持加密传输，故采用 syslog-ng替换syslog，通过TLS实现日志传输的安全性。
n      Syslog-ng软件包
Client ：
http://www.balabit.com/downloads/files/syslog-ng/sources/3.1.0/setups/rhel-5-amd64/syslog-ng-client-3.1.0-1.rhel5.x86_64.rpm
Server ：
http://www.balabit.com/downloads/files/syslog-ng/sources/3.1.0/setups/rhel-4-amd64/syslog-ng-3.1.0-1.rhel4.x86_64.rpm
http://www.balabit.com/downloads/files/syslog-ng/sources/3.1.0/setups/rhel-4-i386/syslog-ng-3.1.0-1.rhel4.i386.rpm
http://www.balabit.com/downloads/files/syslog-ng/sources/3.1.0/setups/rhel-5-i386/syslog-ng-3.1.0-1.rhel5.i386.rpm
 
n      Server端安装 syslog-ng server
1.    使用Rpm命令安装对应操作系统的软件包，安装后系统会自动屏蔽syslog服务的启动
2.    配置OPENSSL证书

mv /etc/init.d/syslog /etc/init.d/remove.syslog
chmod 750 /opt/syslog-ng
cd /opt/syslog-ng/
mkdir ca.d
cd ca.d
echo '100001' >serial
touch index.txt
mkdir private newcerts
vi openssl.cnf
 
Openssl.cnf配置文件如下：

# OpenSSL example configuration file.
# This is mostly being used for generation of certificate requests.
# This definition stops the following lines choking if HOME isn't
# defined.
HOME                     = .
RANDFILE                 = $ENV::HOME/.rnd
 
# Extra OBJECT IDENTIFIER info:
#oid_file                = $ENV::HOME/.oid
oid_section              = new_oids
 
# To use this configuration file with the "-extfile" option of the
# "openssl x509" utility, name here the section containing the
# X.509v3 extensions to use:
# extensions             =
# (Alternatively, use a configuration file that has only
# X.509v3 extensions in its main [= default] section.)
 
[ new_oids ]
 
# We can add new OIDs in here for use by 'ca' and 'req'.
# Add a simple OID like this:
# testoid1=1.2.3.4
# Or use config file substitution like this:
# testoid2=${testoid1}.5.6
 
####################################################################
[ ca ]
default_ca      = CA_default            # The default ca section
 
####################################################################
[ CA_default ]
 
dir              = /opt/syslog-ng/ca.d              # Where everything is kept
certs            = $dir/certs            # Where the issued certs are kept
crl_dir          = $dir/crl              # Where the issued crl are kept
database         = /opt/syslog-ng/ca.d/index.txt # database index file.
#unique_subject = no                     # Set to 'no' to allow creation of
                                        # several ctificates with same subject.
new_certs_dir    = /opt/syslog-ng/ca.d/newcerts          # default place for new certs.
 
certificate      = $dir/cacert.pem       # The CA certificate
serial           = /opt/syslog-ng/ca.d/serial            # The current serial number
crlnumber        = $dir/crlnumber        # the current crl number
                                        # must be commented out to leave a V1 CRL
crl              = $dir/crl.pem          # The current CRL
private_key      = /opt/syslog-ng/ca.d/private/cakey.pem# The private key
RANDFILE         = /opt/syslog-ng/ca.d/private/.rand     # private random number file
 
x509_extensions = usr_cert               # The extentions to add to the cert
# Comment out the following two lines for the "traditional"
# (and highly broken) format.
name_opt         = ca_default            # Subject Name options
cert_opt         = ca_default            # Certificate field options
 
# Extension copying option: use with caution.
# copy_extensions = copy
 
# Extensions to add to a CRL. Note: Netscape communicator chokes on V2 CRLs
# so this is commented out by default to leave a V1 CRL.
# crlnumber must also be commented out to leave a V1 CRL.
# crl_extensions         = crl_ext
 
default_days     = 365                   # how long to certify for
default_crl_days= 30                     # how long before next CRL
default_md       = sha1                  # which md to use.
preserve         = no                    # keep passed DN ordering
 
# A few difference way of specifying how similar the request should look
# For type CA, the listed attributes must be the same, and the optional
# and supplied fields are just that :-)
policy           = policy_match
# For the CA policy
[ policy_match ]
countryName              = match
stateOrProvinceName      = match
organizationName         = match
organizationalUnitName = optional
commonName               = supplied
emailAddress             = optional
 
# For the 'anything' policy
# At this point in time, you must list all acceptable 'object'
# types.
[ policy_anything ]
stateOrProvinceName      = GD
localityName             = optional
organizationName         = Xxxxxxxx
organizationalUnitName = SP
commonName               = supplied
emailAddress             = xxx@xxxx.cn
countryName              = CN
 
####################################################################
[ req ]
default_bits             = 1024
default_md               = sha1
default_keyfile          = privkey.pem
distinguished_name       = req_distinguished_name
attributes               = req_attributes
x509_extensions = v3_ca # The extentions to add to the self signed cert
 
# Passwords for private keys if not present they will be prompted for
# input_password = secret
# output_password = secret
 
# This sets a mask for permitted string types. There are several options.
# default: PrintableString, T61String, BMPString.
# pkix    : PrintableString, BMPString.
# utf8only: only UTF8Strings.
# nombstr : PrintableString, T61String (no BMPStrings or UTF8Strings).
# MASK:XXXX a literal mask value.
# WARNING: current versions of Netscape crash on BMPStrings or UTF8Strings
# so use this option with caution!
# we use PrintableString+UTF8String mask so if pure ASCII texts are used
# the resulting certificates are compatible with Netscape
string_mask = MASK:0x2002
 
# req_extensions = v3_req # The extensions to add to a certificate request
 
[ req_distinguished_name ]
countryName                      = Country Name (2 letter code)
countryName_default              = CN
countryName_min                  = 2
countryName_max                  = 2
 
stateOrProvinceName              = State or Province Name (full name)
stateOrProvinceName_default      = GD
 
localityName                     = Locality Name (eg, city)
localityName_default             = ShenZhen
 
0.organizationName               = Organization Name (eg, company)
0.organizationName_default       = Xxxxxxxx
 
# we can do this but it is not needed normally :-)
#1.organizationName              = Second Organization Name (eg, company)
#1.organizationName_default      = World Wide Web Pty Ltd
 
organizationalUnitName           = Organizational Unit Name (eg, section)
#organizationalUnitName_default =
 
commonName                       = Common Name (eg, your name or your server\'s hostname)
commonName_max                   = 64
 
emailAddress                     = Email Address
emailAddress_max                 = 64
 
# SET-ex3                        = SET extension number 3
 
[ req_attributes ]
challengePassword                = A challenge password
challengePassword_min            = 4
challengePassword_max            = 20
 
unstructuredName                 = An optional company name
 
[ usr_cert ]
 
# These extensions are added when 'ca' signs a request.
 
# This goes against PKIX guidelines but some CAs do it and some software
# requires this to avoid interpreting an end user certificate as a CA.
 
basicConstraints=CA:FALSE
 
# Here are some examples of the usage of nsCertType. If it is omitted
# the certificate can be used for anything *except* object signing.
 
# This is OK for an SSL server.
# nsCertType                     = server
 
# For an object signing certificate this would be used.
# nsCertType = objsign
 
# For normal client use this is typical
# nsCertType = client, email
 
# and for everything including object signing:
# nsCertType = client, email, objsign
 
# This is typical in keyUsage for a client certificate.
# keyUsage = nonRepudiation, digitalSignature, keyEncipherment
 
# This will be displayed in Netscape's comment listbox.
nsComment                        = "OpenSSL Generated Certificate"
 
# PKIX recommendations harmless if included in all certificates.
subjectKeyIdentifier=hash
authorityKeyIdentifier=keyid,issuer
 
# This stuff is for subjectAltName and issuerAltname.
# Import the email address.
# subjectAltName=email:copy
# An alternative to produce certificates that aren't
# deprecated according to PKIX.
# subjectAltName=email:move
 
# Copy subject details
# issuerAltName=issuer:copy
 
#nsCaRevocationUrl               = http://www.domain.dom/ca-crl.pem
#nsBaseUrl
#nsRevocationUrl
#nsRenewalUrl
#nsCaPolicyUrl
#nsSslServerName
 
[ v3_req ]
 
# Extensions to add to a certificate request
 
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
 
[ v3_ca ]
 
 
# Extensions for a typical CA
 
 
# PKIX recommendation.
 
subjectKeyIdentifier=hash
 
authorityKeyIdentifier=keyid:always,issuer:always
 
# This is what PKIX recommends but some broken software chokes on critical
# extensions.
#basicConstraints = critical,CA:true
# So we do this instead.
basicConstraints = CA:true
 
# Key usage: this is typical for a CA certificate. However since it will
# prevent it being used as an test self-signed certificate it is best
# left out by default.
# keyUsage = cRLSign, keyCertSign
 
# Some might want this also
# nsCertType = sslCA, emailCA
 
# Include email address in subject alt name: another PKIX recommendation
# subjectAltName=email:copy
# Copy issuer details
# issuerAltName=issuer:copy
 
# DER hex encoding of an extension: beware experts only!
# obj=DER:02:03
# Where 'obj' is a standard or added object
# You can even override a supported extension:
# basicConstraints= critical, DER:30:03:01:01:FF
 
[ crl_ext ]
 
# CRL extensions.
# Only issuerAltName and authorityKeyIdentifier make any sense in a CRL.
 
# issuerAltName=issuer:copy
authorityKeyIdentifier=keyid:always,issuer:always
 
[ proxy_cert_ext ]
# These extensions should be added when creating a proxy certificate
 
# This goes against PKIX guidelines but some CAs do it and some software
# requires this to avoid interpreting an end user certificate as a CA.
 
basicConstraints=CA:FALSE
 
# Here are some examples of the usage of nsCertType. If it is omitted
# the certificate can be used for anything *except* object signing.
 
# This is OK for an SSL server.
# nsCertType                     = server
 
# For an object signing certificate this would be used.
# nsCertType = objsign
 
# For normal client use this is typical
# nsCertType = client, email
 
# and for everything including object signing:
# nsCertType = client, email, objsign
 
# This is typical in keyUsage for a client certificate.
# keyUsage = nonRepudiation, digitalSignature, keyEncipherment
 
# This will be displayed in Netscape's comment listbox.
nsComment                        = "OpenSSL Generated Certificate"
 
# PKIX recommendations harmless if included in all certificates.
subjectKeyIdentifier=hash
authorityKeyIdentifier=keyid,issuer:always
 
# This stuff is for subjectAltName and issuerAltname.
# Import the email address.
# subjectAltName=email:copy
# An alternative to produce certificates that aren't
# deprecated according to PKIX.
# subjectAltName=email:move
 
# Copy subject details
# issuerAltName=issuer:copy
 
#nsCaRevocationUrl               = http://www.domain.dom/ca-crl.pem
#nsBaseUrl
#nsRevocationUrl
#nsRenewalUrl
#nsCaPolicyUrl
#nsSslServerName
 
# This really needs to be in place for it to be a proxy certificate.
proxyCertInfo=critical,language:id-ppl-anyLanguage,pathlen:3,policy:foo
 
接下来创建证书，注意要输入一致的信息，同时 common name 需要输入服务器的 IP 地址或者域名，这个直接关系到验证是否成功（另外需要确保 server 和 client 的时间一致）
·   Create a CA with a key and generate a certificate request
openssl req -new -config openssl.cnf -keyout private/cakey.pem -out careq.pem

·   Now create a certificate for the CA by using the above key and self sign it.
openssl ca -config openssl.cnf -create_serial -out external-ca.csr -batch -keyfile private/cakey.pem -selfsign -extensions v3_ca -infiles careq.pem

·   Now copy external-ca.csr to cacert.pem because the next command will by default look for the CA certificate with the name cacert.pem
cp external-ca.csr cacert.pem

·   Now create server keys and generate a server certificate request
openssl req -nodes -new -keyout external-server.key -out cert-req.pem -config openssl.cnf

·   Now create a certificate for the server by using the above key and sign it with the CA certificate
openssl ca -config openssl.cnf -out external-server.csr -batch -infiles cert-req.pem

Now we have the following with us
External-ca.csr: CA certificate
External-server.csr: Server certificate which contains the public key
External-server.key: Server private key
 
n       syslog-ng server服务器端配置文件
 
server和client使用TLS验证，但从简化部署的角度，所有的client共用一个证书文件，另外调整了server上的一些参数，启用了流量控制。
 
server端通过2个log配置，将收到的客户端发过来的日志保存到对应主机IP目录下的2个日志文件：
1.       snoopy是用户SHELL操作命令记录
2.       messages是除了snooopy以外的所有日志
注意：需要修改/tmp目录为正式的主日志保存目录，并对这个目录执行chmod 700
 
vi /opt/syslog-ng/etc/syslog-ng.conf
 

@version: 3.0
options {
    flush_lines (0);
    time_reopen (1);
    log_fifo_size (163840);
    long_hostnames(off);
    use_dns (no);
    use_fqdn (no);
    create_dirs (yes);
    keep_hostname (no);
    stats_freq(43200);
    };
 
source src {
    unix-stream("/dev/log");
    internal();
    file("/proc/kmsg" program_override("kernel: "));
    };
 
 
source s_net {
tcp( port(6514)
 
max-connections(300)
log_iw_size(36000)
log_fetch_limit(100)
        tls
        (
                key_file("/opt/syslog-ng/ca.d/external-server.key")
                cert_file("/opt/syslog-ng/ca.d/external-server.csr")
                peer_verify(optional-untrusted)
        )
);
syslog();
};
 
destination d_boot { file("/var/log/boot.log"); };
destination d_cron { file("/var/log/cron.log"); };
destination d_daemon { file("/var/log/daemon.log"); };
destination d_kernel { file("/var/log/kernel.log"); };
destination d_mail { file("/var/log/maillog"); };
destination d_spooler { file("/var/log/spooler"); };
destination d_messages { file("/var/log/messages"); };
destination d_console { usertty("root"); };
destination d_secure { file("/var/log/secure"); };
destination d_fromremote_messages { file(" /tmp/$SOURCEIP/messages.$R_YEAR-$R_MONTH-$R_DAY" log_fifo_size(360000)); };
destination d_fromremote_snoopy { file(" /tmp/$SOURCEIP/snoopy.$R_YEAR-$R_MONTH-$R_DAY" log_fifo_size(360000)); };
 
 
filter f_boot { facility(local7); };
filter f_cron { facility(cron); };
filter f_daemon { facility(daemon); };
filter f_kernel { facility(kern); };
filter f_mail { facility(mail); };
filter f_uucp { facility(cron); };
filter f_news { facility(news); };
filter f_messages { level(info..emerg) and not facility(auth, authpriv, mail, news); };
filter f_emergency { level(emerg); };
filter f_info { level(info); };
filter f_notice { level(notice); };
filter f_warn { level(warn); };
filter f_crit { level(crit); };
filter f_err { level(err); };
filter f_spooler { level(crit...emerg) and facility(news) or facility(uucp); };
filter f_secure { facility(authpriv,auth) and not program(snoopy); };
filter f_notdebug_notmail { level(info...emerg) and not facility(mail); };
 
filter f_snoopy { program(snoopy); };
filter f_not_snoopy { not program(snoopy); };
 
 
log { source(src); filter(f_boot); destination(d_boot); };
log { source(src); filter(f_cron); destination(d_cron); };
log { source(src); filter(f_daemon); destination(d_daemon); };
log { source(src); filter(f_kernel); destination(d_kernel); };
log { source(src); filter(f_mail); destination(d_mail); };
log { source(src); filter(f_messages); destination(d_messages); };
log { source(src); filter(f_emergency); destination(d_console); };
log { source(src); filter(f_secure); destination(d_secure); };
log { source(src); filter(f_spooler); destination(d_spooler); };
 
log { source(s_net); filter(f_snoopy); destination(d_fromremote_snoopy); flags(flow_control); };
log { source(s_net); filter(f_not_snoopy); destination(d_fromremote_messages); flags(flow-control); };
 
/etc/init.d/syslog-ng restart
至此，SYSLOG-NG SERVER 配置完成。
 
 
n       Client 端安装 snoopy

wget http://ncu.dl.sourceforge.net/project/snoopylogger/snoopylogger/snoopy-1.6.1.tar.gz
tar zxvf snoopy-1.6.1.tar.gz
cd snoopy-1.6.1
 ./configure
make
make install
make enable
cd ..
rm -rf snoopy-1.6.1*
 
n       Client 端安装syslog-ng client
1.    使用Rpm命令安装对应操作系统的软件包，安装后系统会自动屏蔽syslog服务的启动

mv /etc/init.d/syslog /etc/init.d/remove.syslog
chmod 750 /opt/syslog-ng
cd /opt/syslog-ng/etc
vi syslog-ng.conf
 
Syslog-ng.conf配置文件如下

@version: 3.0
options {
    flush_lines (0);
    time_reopen (5);
    log_fifo_size (16384);
    long_hostnames(off);
    use_dns (no);
    use_fqdn (no);
    create_dirs (yes);
    keep_hostname (no);
    stats_freq(43200);
    };
 
source src {
    unix-stream("/dev/log" log_fetch_limit(100) log_iw_size(1000));
    internal();
    file("/proc/kmsg" program_override("kernel: ") log_fetch_limit(100) log_iw_size(1000));
    };
 
 
destination d_boot { file("/var/log/boot.log"); };
destination d_cron { file("/var/log/cron.log"); };
destination d_daemon { file("/var/log/daemon.log"); };
destination d_kernel { file("/var/log/kernel.log"); };
destination d_mail { file("/var/log/maillog"); };
destination d_spooler { file("/var/log/spooler"); };
destination d_messages { file("/var/log/messages"); };
destination d_console { usertty("root"); };
destination d_secure { file("/var/log/secure"); };
 
 
destination d_remote {
                tcp("192.168.168.122" port(6514)
                    tls( ca_dir("/opt/syslog-ng/etc/")) ); };
 
 
filter f_boot { facility(local7); };
filter f_cron { facility(cron); };
filter f_daemon { facility(daemon); };
filter f_kernel { facility(kern); };
filter f_mail { facility(mail); };
filter f_uucp { facility(cron); };
filter f_news { facility(news); };
filter f_messages { level(info..emerg) and not facility(auth, authpriv, mail, news); };
filter f_emergency { level(emerg); };
filter f_info { level(info); };
filter f_notice { level(notice); };
filter f_warn { level(warn); };
filter f_crit { level(crit); };
filter f_err { level(err); };
filter f_spooler { level(crit...emerg) and facility(news) or facility(uucp); };
filter f_secure { facility(authpriv,auth) and not program(snoopy); };
filter f_notdebug_notmail { level(info...emerg) and not facility(mail); };
 
log { source(src); filter(f_boot); destination(d_boot); };
log { source(src); filter(f_cron); destination(d_cron); };
log { source(src); filter(f_daemon); destination(d_daemon); };
log { source(src); filter(f_kernel); destination(d_kernel); };
log { source(src); filter(f_mail); destination(d_mail); };
log { source(src); filter(f_messages); destination(d_messages); };
log { source(src); filter(f_emergency); destination(d_console); };
log { source(src); filter(f_secure); destination(d_secure); };
log { source(src); filter(f_spooler); destination(d_spooler); };
log { source(src); filter(f_notdebug_notmail); destination(d_remote); flags(flow-control);};
 
将前面在Server上用openssl命令产生的external-ca.csr文件COPY到syslog-ng客户端的/opt/syslog-ng/etc目录下，然后执行下面的命令：
openssl x509 –noout –hash –in external-ca.csr
·   The output of the above command is a hash value and will be of the format fa6084d0. Now create a symbolic link for the certificate for debugging purposes.
ln –s external-ca.csr fa6084d0.0
(please note the .0 suffix)

 
/etc/init.d/syslog-ng restart
至此，SYSLOG-NG client      配置完成。
 
 
 
n      Server Debug
使用下面的命令启动 syslog-ng ，即可在控制台查看详细 DEBUG 信息
/opt/syslog-ng/sbin/syslog-ng -e -F -d -v
 
 
n      关于 Flow-control
 
为了启用流量控制，Flags(flow-control)必须在log对象里面激活，并通过source/destination的几个属性控制。
For example:
log { source(s_localhost); destination(d_tcp); flags(flow-control); };
source:
log_iw_size > max_connections() *log_fetch_limit
log_fetch_limit() (default value: 10)
(每个source上独立连接，对于TCP and unix-stream sources, syslog-ng从每个source的每个连接读取log_fetch_limit() 条消息
log_iw_size() (default value: 100)
Flow-control使用一个control window来决定输出buffer里面是否有空闲空间为新的messages使用.
每个source有自己的control window-- log_iw_size() parameter
当一个source容许多个连接，每个连接的消息使用同一个control window.
如果control window 满了, syslog-ng 停止从source读取消息直到一些消息成功发送给destination.
destination:
log_fifo_size > log_iw_size *10
每个destination有一个输出buffer (log_fifo_size()).The log_fifo_size() parameter sets the size of the output buffer.
输出buffer必须大于每个SOURCE发送给destination的control window
如果output buffer满了，并且disk-buffering 和 flow-control 都没有使用, messages may be lost.
 
n       Replacing klogd on Linux
The syslog-ng application can replace both the syslogd and klogd daemons on Linux hosts. To replace klogd, complete the following steps:
 
 
Add a file source pointing to /proc/kmsg to the syslog-ng configuration file.
 
source s_kmsg { file("/proc/kmsg"); };
 
 Warning
Do not use a pipe source to read /proc/kmsg; pipe opens the source in read-write mode and this may cause problems when using SELinux or similar security measures.
 
 
Include the source defined in Step 1 in a log path.
 
Stop klogd.
 
 
Do not run klogd and syslog-ng simultaneously when using syslog-ng to read /proc/kmsg, as it might block syslog-ng.