Python使用yaml配置logging日志

前言

在Python环境下，logging是一个很好的用于处理日志的扩展包。平时使用时一般直接import logging后，调用logging.debug(“info”)方法打印日志。一般情况下够用，但是在需要涉及到复杂的日志功能（例如分级、写文件、归档等）后，则需要深入地对logging进行配置。通常情况下会使用yaml配置。

实践

创建一个loggingconfigyaml.yaml文件，内容如下：

version: 1
disable_existing_loggers: False

formatters:
    simple:
        format: '%(asctime)s %(levelname)s %(message)s'
    upgrade:
        format: "%(asctime)s -Loc %(filename)s -Pid %(process)d -%(name)s -%(levelname)s - %(message)s"

handlers:
    console:
        class: logging.StreamHandler
        level: DEBUG
        formatter: simple
        stream: ext://sys.stdout

    all_file_handler:
        class: logging.handlers.RotatingFileHandler
        level: DEBUG
        formatter: upgrade
        filename: ./logs/all_logs/all_log.log
        maxBytes: 10485760 # 10MB
        backupCount: 50 #保留50个log文件
        encoding: utf8
    
    server_file_handler:
        class: logging.handlers.RotatingFileHandler
        level: INFO # 只在文件中记录INFO级别及以上的log
        formatter: upgrade
        filename: ./logs/server_logs/server.log
        maxBytes: 10485760 # 10MB
        backupCount: 20
        encoding: utf8

loggers:
    server:
        level: DEBUG #允许打印DEBUG及以上log
        handlers: [server_file_handler]
        propagate: true #设为false则禁止将日志消息传递给父级记录器的处理程序中

root:
    level: DEBUG
    handlers: [console, all_file_handler]

通过Python的yaml包读取并加载配置，对logging应用该配置。

import yaml
import logging.config
import os

def test():
    with open(file="loggingconfigyaml.yaml", mode='r', encoding="utf-8")as file:
        logging_yaml = yaml.load(stream=file, Loader=yaml.FullLoader)
    handlers = logging_yaml['handlers']
    for key, value in handlers.items():
        if 'filename' in value:
            log_path = (os.path.split(value['filename'])[0])
            if not os.path.exists(log_path):
                os.makedirs(log_path)
    # 配置logging日志：主要从文件中读取handler的配置、formatter（格式化日志样式）、logger记录器的配置
    logging.config.dictConfig(config=logging_yaml)
    ###设置完毕###
    # 获取根记录器：配置信息从yaml文件中获取
    root = logging.getLogger()
    # 子记录器的名字与配置文件中loggers字段内的保持一致
    server = logging.getLogger("server")
    print("rootlogger:", root.handlers)
    print("serverlogger:", server.handlers)
    print("子记录器与根记录器的handler是否相同：", root.handlers[0] == server.handlers[0])

if __name__ == '__main__':
    test()

高阶使用推荐

带色彩的日志配置，移步gist.github - kingspp/logging.py：

logging.py：

import os
import yaml
import logging.config
import logging
import coloredlogs

def setup_logging(default_path='logging.yaml', default_level=logging.INFO, env_key='LOG_CFG'):
    """
    | **@author:** Prathyush SP
    | Logging Setup
    """
    path = default_path
    value = os.getenv(env_key, None)
    if value:
        path = value
    if os.path.exists(path):
        with open(path, 'rt') as f:
            try:
                config = yaml.safe_load(f.read())
                logging.config.dictConfig(config)
                coloredlogs.install()
            except Exception as e:
                print(e)
                print('Error in Logging Configuration. Using default configs')
                logging.basicConfig(level=default_level)
                coloredlogs.install(level=default_level)
    else:
        logging.basicConfig(level=default_level)
        coloredlogs.install(level=default_level)
        print('Failed to load configuration file. Using default configs')

logging.yaml：

version: 1
disable_existing_loggers: true



formatters:
    standard:
        format: "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
    error:
        format: "%(levelname)s <PID %(process)d:%(processName)s> %(name)s.%(funcName)s(): %(message)s"

handlers:
    console:
        class: logging.StreamHandler
        level: DEBUG
        formatter: standard
        stream: ext://sys.stdout

    info_file_handler:
        class: logging.handlers.RotatingFileHandler
        level: INFO
        formatter: standard
        filename: /tmp/info.log
        maxBytes: 10485760 # 10MB
        backupCount: 20
        encoding: utf8

    error_file_handler:
        class: logging.handlers.RotatingFileHandler
        level: ERROR
        formatter: error
        filename: /tmp/errors.log
        maxBytes: 10485760 # 10MB
        backupCount: 20
        encoding: utf8

    debug_file_handler:
        class: logging.handlers.RotatingFileHandler
        level: DEBUG
        formatter: standard
        filename: /tmp/debug.log
        maxBytes: 10485760 # 10MB
        backupCount: 20
        encoding: utf8

    critical_file_handler:
        class: logging.handlers.RotatingFileHandler
        level: CRITICAL
        formatter: standard
        filename: /tmp/critical.log
        maxBytes: 10485760 # 10MB
        backupCount: 20
        encoding: utf8

    warn_file_handler:
        class: logging.handlers.RotatingFileHandler
        level: WARN
        formatter: standard
        filename: /tmp/warn.log
        maxBytes: 10485760 # 10MB
        backupCount: 20
        encoding: utf8

root:
    level: NOTSET
    handlers: [console]
    propogate: yes

loggers:
    <module>:
        level: INFO
        handlers: [console, info_file_handler, error_file_handler, critical_file_handler, debug_file_handler, warn_file_handler]
        propogate: no

    <module.x>:
        level: DEBUG
        handlers: [info_file_handler, error_file_handler, critical_file_handler, debug_file_handler, warn_file_handler]
        propogate: yes

参考资料

[1] logging — Python 的日志记录工具
[2] gist.github - kingspp/logging.py
[3] python之使用yaml文件配置logging日志
[4] coloredlogs