服务器审计日志备份python脚本开发

---

前言

在项目验收阶段反馈服务器未开启审计日志以及该日志没有做备份处理,不满足等保要求,必须整改。前提是客户不愿意购买第三方审计系统,同时对服务器资源也非常限制，在此种情况下，三方会谈后确认将所有服务器的审计日志备份到当前环境的跳板机服务器中，保留30天,超过30天则删除最早的那一次备份。在此种情况下进行备份脚本的开发，其实shell也可以简单实现,为了加强python语法，因此该脚本功能使用python开发实现

python版本3.6.8, 服务器系统centos7.6

---

一、脚本功能

#该 AuditLogSync.py 脚本主要作用
1、用于等保测评,将服务器上的审计日志通过多线程脚本方式打包后同步到 10.10.10.10 服务器上,实现审计日志备份功能。(前提是服务器开启审计日志功能。同时该脚本还能实现自动删除 7 天前的审计日志归档目录,只保留 7 天的归档记录。避免占用太多的磁盘空间)
2、支持服务器密码一致和不一致两种形式,都可以实现备份功能

#执行步骤：
1、将 IP 列表写入 hosts 文件中,与脚本同一目录下即可,注意 hosts 文件形式,支持以下两种形式:
    a、当服务器密码一致时,则只在 hosts 文件中添加服务器 ip 即可,通过代码中的 parser 指定了默认密码
    b、当服务器密码不一致时,则 hosts 文件中的形式是"ip,密码",通过代码获取 ip、密码

2、执行命令 python3 -W ignore AuditLogSync.py (-W ignore 主要是忽略掉执行时产生的一个 Warning 提醒,与脚本无关)
注意:
    因为这个提醒是 python3.6 不再被支持,因此才会有这个提醒,最完美的解决方法是修改 python3.6 中这个文件/usr/local/lib64/python3.6/site-packages/cryptography/__init__.py,注释掉以下即可
    #if sys.version_info[:2] == (3, 6):
    #    warnings.warn(
    #        "Python 3.6 is no longer supported by the Python core team. "
    #        "Therefore, support for it is deprecated in cryptography. The next "
    #        "release of cryptography will remove support for Python 3.6.",
    #        CryptographyDeprecationWarning,
    #        stacklevel=2,
    #)

3、执行完成后,会在脚本同级目录下产生 sync_error.log 和 sync_info.log 日志文件,到时查看日志即可

4、审计日志归档目录位置/data/AuditLogSync/xx(年月日命名的目录)

补充:
    1、将脚本构建为二进制程序文件,保护源码
        [root@ops23 AuditLogSync]# /usr/local/bin/pyinstaller --onefile --clean  AuditLogSync.py
    2、二进制程序运行
        [root@ops23 AuditLogSync]# /data/AuditLogSync/AuditLogSync.py --port 43322

二、脚本开发

1.脚本示例

脚本如下：

[root@ops23 AuditLogSync]# vim AuditLogSync.py
#!/usr/bin/env python3
#Author: ops
#Date: 202412172235
import os
import time
import paramiko
import logging
from datetime import datetime
import threading
import inspect
import argparse

class AuditLogSync:
    def __init__(self, ip_file,audit_path,username,password,port):
        """
        初始化AuditLogSync类

        Parameters:
        ip_file (str): 包含IP地址列表的文件路径
        audit_path (str): 审计日志路径
        username (str): 用户名
        password (str): 密码
        port    (int):  端口
        """
        self.ip_file = ip_file
        self.audit_path=audit_path
        self.username = username
        self.password = password
        self.port = port
        #生成日期
        self.current_time = datetime.now().strftime("%Y%m%d")
        #生成审计日志归档目录
        self.export_dir = f"/data/AuditLogSync/{self.current_time}"
        #生成日志文件
        self.info_log_file = "/data/AuditLogSync/sync_info.log"
        self.error_log_file = "/data/AuditLogSync/sync_error.log"
        # 设置 info 日志记录器
        self.info_logger = logging.getLogger('info_logger')
        info_handler = logging.FileHandler(self.info_log_file)
        info_handler.setLevel(logging.INFO)
        info_formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
        info_handler.setFormatter(info_formatter)
        self.info_logger.addHandler(info_handler)
        self.info_logger.setLevel(logging.INFO)
        # 设置 error 日志记录器
        self.error_logger = logging.getLogger('error_logger')
        error_handler = logging.FileHandler(self.error_log_file)
        error_handler.setLevel(logging.ERROR)
        error_formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
        error_handler.setFormatter(error_formatter)
        self.error_logger.addHandler(error_handler)
        #判断审计日志归档目录是否存在
        if not os.path.exists(self.export_dir):
            os.makedirs(self.export_dir)
    
   
    def run_operations(self):
        """
        读取服务器列表文件,解析ip、密码,开启线程操作
        """
        with open(self.ip_file, 'r') as file:
            ip_password_pairs = file.readlines()
        
        threads = []
        for ip_password_pair in ip_password_pairs:
            ip_address, password = self.parse_ip_password(ip_password_pair.strip())
            thread = threading.Thread(target=self.process_server, args=(ip_address, password))
            threads.append(thread)
            thread.start()
        
        for thread in threads:
            thread.join()
    
    def parse_ip_password(self, ip_password_pair):
        """
        解析IP地址和密码

        Parameters:
        ip_password_pair (str): IP地址和密码的组合字符串

        Returns:
        tuple: (ip_address, password)
        """
        if ',' in ip_password_pair:
            ip_address, password = ip_password_pair.split(',', 1)
        else:
            ip_address = ip_password_pair
            password = self.password
        return ip_address, password

    def process_server(self, ip_address,password):
        """
        处理单个服务器的操作
        """
        ssh_client = self.connect_to_server(ip_address,password)
        #判断服务器连接是否成功,如果成功调用打包审计日志方法、执行rsync同步方法，最后关闭连接
        if ssh_client is not None:
            self.archive_logs(ssh_client, ip_address)
            self.execute_rsync(ssh_client, ip_address)
            if ip_address == "10.10.10.10":
               self.delete_old_archive_logs(ssh_client)
            ssh_client.close()
    def connect_to_server(self, ip_address,password):
        """
        建立SSH连接到远程服务器

        Parameters:
        ip_address (str): 目标服务器的IP地址

        Returns:
        paramiko.SSHClient: SSH客户端对象
        """
        ssh_client = paramiko.SSHClient()
        ssh_client.set_missing_host_key_policy(paramiko.AutoAddPolicy())

        try:
            # 尝试连接到服务器
            ssh_client.connect(ip_address, username=self.username, password=password, port=self.port,timeout=20)
            self.info_logger.info(f"代码块: {inspect.currentframe().f_code.co_name} - 成功连接到服务器 {ip_address}")
        except paramiko.ssh_exception.AuthenticationException as auth_err:
            # 认证失败错误处理
            self.error_logger.error(f"代码块: {inspect.currentframe().f_code.co_name} - 认证失败连接服务器 {ip_address}: {auth_err} <请检查服务器信息是否正确>")
            return None
        except paramiko.ssh_exception.NoValidConnectionsError as conn_err:
            # 连接失败错误处理
            self.error_logger.error(f"代码块: {inspect.currentframe().f_code.co_name} - 无法连接到服务器 {ip_address}: {conn_err} <请检查服务器网络或端口设置>")
            return None
        except Exception as e:
            # 处理其他未知异常
            self.error_logger.error(f"代码块: {inspect.currentframe().f_code.co_name} - 连接到 {ip_address} 发生未知错误: {e}")
            return None

        return ssh_client

    def archive_logs(self, ssh_client, ip_address):
        """
        打包远程服务器上的日志文件

        Parameters:
        ssh_client (paramiko.SSHClient): SSH客户端对象
        ip_address (str): 目标服务器的IP地址
        """
        check_dir_cmd = f"""
        if [ -d {self.audit_path} ]; then
            echo 'Directory exists'
        else
           echo 'Directory does not exist'
        fi
      """
        stdin, stdout, stderr = ssh_client.exec_command(check_dir_cmd)
        dir_check_output = stdout.read().decode('utf-8')
        if "Directory exists" in dir_check_output:
           self.info_logger.info(f"代码块: {inspect.currentframe().f_code.co_name} - {ip_address}: 审计日志目录 {self.audit_path} 存在")
           tar_cmd = f"cd {self.audit_path} && tar -zcf /tmp/{ip_address}_{self.current_time}_audit.tar.gz ."
           stdin, stdout, stderr = ssh_client.exec_command(tar_cmd)
           exit_status = stdout.channel.recv_exit_status()
           # 判断打包是否成功
           if exit_status == 0:
              self.info_logger.info(f"代码块: {inspect.currentframe().f_code.co_name} - {ip_address}: 审计日志打包成功")
           else:
              self.error_logger.error(f"代码块: {inspect.currentframe().f_code.co_name} - {ip_address}: 审计日志打包失败 - 审计日志打包失败详情: {stderr.read().decode('utf-8')}")
        else:
           self.error_logger.error(f"代码块: {inspect.currentframe().f_code.co_name} - {ip_address}: 审计日志目录 {self.audit_path} 不存在 - 详情: {stderr.read().decode('utf-8')}")
    def execute_rsync(self, ssh_client, ip_address):
        """
        执行rsync同步操作

        Parameters:
        ssh_client (paramiko.SSHClient): SSH客户端对象
        ip_address (str): 目标服务器的IP地址
        """
        rsync_check_cmd = "which rsync"
        _, stdout, stderr = ssh_client.exec_command(rsync_check_cmd)
        #调试
        #print(f"execute_rsync_stdout: {stdout.read().decode('utf-8')}")
        if "rsync" not in stdout.read().decode('utf-8'):
            install_rsync_cmd = "yum -y install rsync"
            ssh_client.exec_command(install_rsync_cmd)
            #安装rsync命令可能需要20s左右
            self.info_logger.info(f"{ip_address}: 安装rsync中,请等待... - {inspect.currentframe().f_code.co_name}")
            time.sleep(20)
            # 再次检查rsync命令是否安装成功
            stdin, stdout, stderr = ssh_client.exec_command(rsync_check_cmd)
            if "rsync" not in stdout.read().decode('utf-8'):
                self.error_logger.error(f"代码块: {inspect.currentframe().f_code.co_name} - {ip_address}: 安装rsync失败 - 失败详情: {stderr.read().decode('utf-8')}")
                return
        # 已经安装rsync，执行目标服务器的操作
        rsync_cmd = f"rsync -az -e 'ssh -p {self.port}' {self.username}@{ip_address}:/tmp/{ip_address}_{self.current_time}_audit.tar.gz {self.export_dir}"
        os.system(rsync_cmd)
        self.info_logger.info(f"代码块: {inspect.currentframe().f_code.co_name} - {ip_address}: 审计日志压缩包同步成功")
        # 删除远程服务器上的tar包
        delete_tar_cmd = f"rm /tmp/{ip_address}_{self.current_time}_audit.tar.gz"
        _, stdout, stderr = ssh_client.exec_command(delete_tar_cmd)
        if stderr.read().decode('utf-8'):
            self.error_logger.error(f"代码块: {inspect.currentframe().f_code.co_name} - {ip_address}: 删除远程服务器上的tar包失败 - 失败详情: {stderr.read().decode('utf-8')}")
            return

        self.info_logger.info(f"代码块: {inspect.currentframe().f_code.co_name} - {ip_address}: 成功删除远程服务器上的tar包")
 
    def delete_old_archive_logs(self, ssh_client):
        # 删除旧归档日志目录方法
        check_archive_cmd = f"find /data/AuditLogSync/ -mindepth 1 -maxdepth 1  -type d -mtime +7"
        _, stdout, stderr = ssh_client.exec_command(check_archive_cmd)
        if stderr.read().decode('utf-8'):
            self.error_logger.error(f"代码块: {inspect.currentframe().f_code.co_name} - 执行文件个数命令失败 - 失败详情: {stderr.read().decode('utf-8')}")
            return
        old_archive_dirs = stdout.read().decode('utf-8').splitlines()
        for old_dir in old_archive_dirs:
            delete_cmd = f"rm -rf {old_dir}"
            _, stderr, _ = ssh_client.exec_command(delete_cmd)
            if  stderr.read().decode('utf-8'):
               self.error_logger.error(f"代码块: {inspect.currentframe().f_code.co_name} - 10.10.10.10 删除旧归档日志目录失败 - 旧归档日志目录删除失败详情: {stderr.read().decode('utf-8')}")
            else:
               self.info_logger.info(f"代码块: {inspect.currentframe().f_code.co_name} - 10.10.10.10 删除旧归档日志目录成功: {old_dir}")

# 使用AuditLogSync类来运行操作,程序执行入口
if __name__ == '__main__':
    # 设置命令行参数解析
    parser = argparse.ArgumentParser(
    	description='服务器审计日志备份工具',
    	formatter_class=argparse.ArgumentDefaultsHelpFormatter
    )
    #如果以下几个参数必须要在命令行指定,则将False替换为True
    parser.add_argument('--ip-file',required=False,help='服务器列表文件',default='/data/AuditLogSync/hosts')  #修改
    parser.add_argument('--audit-path', required=False, help='需要备份的审计日志目录',default='/var/log/audit')  #修改
    parser.add_argument('--username', required=False, help='服务器默认用户',default="root")    #修改
    parser.add_argument('--password', required=False, help='服务器默认密码',default="dddddd") #修改
    parser.add_argument('--port', type=int, required=True, help='服务器默认端口',default=22)   #修改

    # 解析命令行参数
    args = parser.parse_args()
    # 调试打印命令行参数和默认值
    #print("解析后的命令行参数:")
    #print(f"ip-file: {args.ip_file}")
    #print(f"audit-path: {args.audit_path}")
    #print(f"username: {args.username}")
    #print(f"password: {args.password}")
    #print(f"port: {args.port}")
    # 使用命令行参数创建AuditLogSync实例
    auditLogSync = AuditLogSync(args.ip_file, args.audit_path, args.username, args.password, args.port)
    auditLogSync.run_operations()

2.执行示例

执行过程如下（示例）：

2.1.安装构建二进制程序模块

[root@ops23 AuditLogSync]#  pip3 install pyinstaller -i https://pypi.tuna.tsinghua.edu.cn/simple

2.2.执行构建命令、展示构建过程,具体的参数含义请自行查阅百度

[root@ops23 AuditLogSync]# /usr/local/bin/pyinstaller --onefile --clean  AuditLogSync.py
278 INFO: PyInstaller: 4.10
278 INFO: Python: 3.6.8
280 INFO: Platform: Linux-3.10.0-693.el7.x86_64-x86_64-with-centos-7.4.1708-Core
281 INFO: wrote /data/AuditLogSync/AuditLogSync.spec
286 INFO: UPX is not available.
287 INFO: Removing temporary files and cleaning cache in /root/.cache/pyinstaller
289 INFO: Extending PYTHONPATH with paths
['/data/AuditLogSync']
737 INFO: checking Analysis
737 INFO: Building Analysis because Analysis-00.toc is non existent
737 INFO: Initializing module dependency graph...
739 INFO: Caching module graph hooks...
755 INFO: Analyzing base_library.zip ...
7086 INFO: Caching module dependency graph...
7426 INFO: running Analysis Analysis-00.toc
7463 INFO: Analyzing /export/AuditLogSync/AuditLogSync.py
10452 INFO: Processing module hooks...
10452 INFO: Loading module hook 'hook-bcrypt.py' from '/usr/local/lib/python3.6/site-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
10453 INFO: Loading module hook 'hook-cryptography.py' from '/usr/local/lib/python3.6/site-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
10955 INFO: Loading module hook 'hook-nacl.py' from '/usr/local/lib/python3.6/site-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
10972 INFO: Loading module hook 'hook-heapq.py' from '/usr/local/lib/python3.6/site-packages/PyInstaller/hooks'...
10975 INFO: Loading module hook 'hook-pickle.py' from '/usr/local/lib/python3.6/site-packages/PyInstaller/hooks'...
10980 INFO: Loading module hook 'hook-xml.py' from '/usr/local/lib/python3.6/site-packages/PyInstaller/hooks'...
11434 INFO: Loading module hook 'hook-difflib.py' from '/usr/local/lib/python3.6/site-packages/PyInstaller/hooks'...
11437 INFO: Loading module hook 'hook-encodings.py' from '/usr/local/lib/python3.6/site-packages/PyInstaller/hooks'...
11575 INFO: checking Tree
11576 INFO: Building Tree because Tree-00.toc is non existent
11576 INFO: Building Tree Tree-00.toc
11577 INFO: Looking for ctypes DLLs
11578 INFO: Analyzing run-time hooks ...
11585 INFO: Including run-time hook '/usr/local/lib/python3.6/site-packages/PyInstaller/hooks/rthooks/pyi_rth_subprocess.py'
11588 INFO: Including run-time hook '/usr/local/lib/python3.6/site-packages/PyInstaller/hooks/rthooks/pyi_rth_pkgutil.py'
11593 INFO: Including run-time hook '/usr/local/lib/python3.6/site-packages/PyInstaller/hooks/rthooks/pyi_rth_inspect.py'
11611 INFO: Looking for dynamic libraries
12884 INFO: Looking for eggs
12884 INFO: Using Python library /lib64/libpython3.6m.so.1.0
12893 INFO: Warnings written to /data/AuditLogSync/build/AuditLogSync/warn-AuditLogSync.txt
12971 INFO: Graph cross-reference written to /data/AuditLogSync/build/AuditLogSync/xref-AuditLogSync.html
12988 INFO: checking PYZ
12988 INFO: Building PYZ because PYZ-00.toc is non existent
12989 INFO: Building PYZ (ZlibArchive) /data/AuditLogSync/build/AuditLogSync/PYZ-00.pyz
14079 INFO: Building PYZ (ZlibArchive) /data/AuditLogSync/build/AuditLogSync/PYZ-00.pyz completed successfully.
14091 INFO: checking PKG
14092 INFO: Building PKG because PKG-00.toc is non existent
14092 INFO: Building PKG (CArchive) AuditLogSync.pkg
24507 INFO: Building PKG (CArchive) AuditLogSync.pkg completed successfully.
24512 INFO: Bootloader /usr/local/lib/python3.6/site-packages/PyInstaller/bootloader/Linux-64bit-intel/run
24513 INFO: checking EXE
24513 INFO: Building EXE because EXE-00.toc is non existent
24513 INFO: Building EXE from EXE-00.toc
24513 INFO: Copying bootloader EXE to /data/AuditLogSync/dist/AuditLogSync
24513 INFO: Appending PKG archive to custom ELF section in EXE
24593 INFO: Building EXE from EXE-00.toc completed successfully.

最终构建完成后会产生三个目录,分别是dist、build、__python__,构建好的程序在dist目录下,从该目录下移出来二进制程序文件,其余可以删除掉

2.3.指定hosts文件

2.4.执行备份命令

2.4.1.查看备份帮助命令

[root@ops23 AuditLogSync]#  ./AuditLogSync -h

2.4.2.执行命令

后续改动只需要添加参数及对应值即可

[root@ops23 AuditLogSync]# ./AuditLogSync --port 43322

如果要使用定时任务，如下操作

[root@ops23 AuditLogSync]# crontab -e
#AuditLogSync
55 23 * * * /data/AuditLogSync/AuditLogSync --port 33822

2.5.审计日志备份结果检查

因为我们是将所有审计日志备份到10.10.10.10这台机器的/data/AuditLogSync/目录下,以日期目录为主，因此检查该服务的目录

[root@ops23 AuditLogSync]# ll

检查产生的日志文件

[root@ops23 AuditLogSync]# vim sync_info.log
[root@ops23 AuditLogSync]# vim sync_error.log

至此,服务器审计日志备份完成

2.6.整体目录

---

总结

给大家分享一个关于python3.6版本中argparse命令行模块的使用方法地址https://docs.python.org/zh-cn/3.6/library/argparse.html#formatter-class，刚好该脚本中就使用到了这个模块,可以结合着该文档容易理解argparse模块这部分代码