Python日志库logging、loguru、Eliot

简介

logging，Python内置库，实现了事件日志系统的函数与类。

loguru，第三方库，轻松记日志，一个函数搞定。

Eliot，一款强大的日志系统，告诉你：性能瓶颈、什么时候发生、谁调用了什么

初试

日志默认级别是 WARNING，所以 INFO 信息没有出现

import logging

logging.info('普通信息')
logging.warning('警告！')
# WARNING:root:警告！

记录 DEBUG 及以上级别日志，保存到 test.log

import logging

logging.basicConfig(filename='test.log', level=logging.DEBUG)  # 记录DEBUG及以上级别日志
logging.debug('调试消息')
logging.info('普通消息')
logging.warning('警告消息')
logging.error('错误消息')
logging.critical('严重错误消息')
try:
    1 / 0
except Exception as e:
    logging.exception(e)

效果

日志基础教程

日志级别（以严重性递增）

级别	含义
DEBUG	调试信息，诊断问题时用
INFO	确认程序预期运行
WARNING	警告，如磁盘空间不足，程序仍进行
ERROR	错误，程序某些功能已不能正常进行
CRITICAL	严重错误，程序已不能继续进行

什么时候使用日志

任务	工具
结果显示在控制台的程序	print()
发生普通操作时提交事件报告(如：状态监控和错误调查)	logging.info()，需要详细输出时用 logging.debug()
特殊运行时警告	warnings.warn() 或 logging.warning()
特殊运行时报告错误	引发异常
报告错误而不引发异常(如长时间运行的服务端进程)	logging.error(), logging.exception() 或 logging.critical() 分别适用于特定的错误及应用领域

消息格式

日志级别: 消息

import logging

logging.basicConfig(format='%(levelname)s: %(message)s', level=logging.DEBUG)
logging.debug('调试消息')
logging.info('普通消息')
logging.warning('警告消息')
logging.error('错误消息')
logging.critical('严重错误消息')

输出

DEBUG: 调试信息
INFO: 普通信息
WARNING: 警告信息
ERROR: 错误信息
CRITICAL: 严重错误信息

时间 消息

import logging

logging.basicConfig(format='%(asctime)s %(message)s', level=logging.DEBUG, datefmt='%Y-%m-%d %H:%M:%S')
logging.debug('调试消息')
logging.info('普通消息')
logging.warning('警告消息')
logging.error('错误消息')
logging.critical('严重错误消息')

输出

2020-07-21 15:00:00 调试消息
2020-07-21 15:00:00 普通消息
2020-07-21 15:00:00 警告消息
2020-07-21 15:00:00 错误消息
2020-07-21 15:00:00 严重错误消息

查看更多时间格式

日志属性

查看更多日志属性

import logging

# 时间
# logging.basicConfig(format='%(created)f %(message)s')  # 被创建的时间，即 time.time()。1595317659.993426 xxx
# logging.basicConfig(format='%(asctime)s %(message)s', datefmt='%Y-%m-%d %H:%M:%S')  # 供人查看的时间。2020-07-21 15:47:54 xxx
# logging.basicConfig(format='%(msecs)d %(message)s')  # 创建时间毫秒部分。655 xxx
# logging.basicConfig(format='%(relativeCreated)d %(message)s')  # 相对于 logging 加载的时间差。289 xxx

# 文件名
# logging.basicConfig(format='%(filename)s %(message)s')  # 文件名。test.py xxx
# logging.basicConfig(format='%(module)s %(message)s')  # 模块名。test xxx
# logging.basicConfig(format='%(name)s %(message)s')  # 日志记录器名。root xxx
# logging.basicConfig(format='%(pathname)s %(message)s')  # 完整路径名。D:/code/test/test.py xxx

logging.basicConfig(format='%(funcName)s %(message)s')  # 函数名。f xxx
def f():
    logging.warning('xxx')
f()

# 消息级别
# logging.basicConfig(format='%(levelname)s %(message)s')  # 消息级别。WARNING xxx
# logging.basicConfig(format='%(levelno)s %(message)s')  # 消息数字级别。30 xxx

# 行号
# logging.basicConfig(format='%(lineno)d %(message)s')  # 消息数字级别。21 xxx

# 进程和线程
# logging.basicConfig(format='%(process)d %(message)s')  # 进程ID。100240 xxx
# logging.basicConfig(format='%(processName)s %(message)s')  # 进程名。MainProcess xxx
# logging.basicConfig(format='%(thread)d %(message)s')  # 线程ID。131596 xxx
# logging.basicConfig(format='%(threadName)s %(message)s')  # 线程名。MainThread xxx
logging.warning('xxx')

信息流程

logging库主要通过模块化的方法记录日志，主要模块有：

• Logger，记录器，提供直接调用的接口。
• Handler，处理器，将Logger创建的日志记录发送到合适的地方。
• Filter，过滤器，提供更精细的功能，用于确定要输出的日志记录。
• Formatter，格式器，指定最终输出的样式。

import logging

logger = logging.getLogger('test')  # 创建logger
logger.setLevel(logging.DEBUG)
ch = logging.StreamHandler()  # 创建handler
ch.setLevel(logging.DEBUG)
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s', datefmt='%Y-%m-%d %H:%M:%S')  # 创建formatter
ch.setFormatter(formatter)  # 为handler设置formatter
logger.addHandler(ch)  # 为logger添加handler

if __name__ == '__main__':
    logger.debug('调试消息')
    logger.info('普通消息')
    logger.warning('警告消息')
    logger.error('错误消息')
    logger.critical('严重错误消息')

通过配置文件创建

实现配置和代码的分离，详细查阅日志记录配置

logging.ini

[loggers]
keys=root,test

[handlers]
keys=consoleHandler

[formatters]
keys=simpleFormatter

[logger_root]
level=DEBUG
handlers=consoleHandler

[logger_test]
level=DEBUG
handlers=consoleHandler
qualname=test
propagate=0

[handler_consoleHandler]
class=StreamHandler
level=DEBUG
formatter=simpleFormatter
args=(sys.stdout,)

[formatter_simpleFormatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
datefmt=%Y-%m-%d %H:%M:%S

代码

import logging
import logging.config

logging.config.fileConfig('logging.ini')  # 加载配置
logger = logging.getLogger('test')  # 创建logger

if __name__ == '__main__':
    logger.debug('调试消息')
    logger.info('普通消息')
    logger.warning('警告消息')
    logger.error('错误消息')
    logger.critical('严重错误消息')

PyCharm日志插件

安装方法：File → Settings → Plugins → Marketplace 搜 Ideolog → Install → Restart

需要根据正则表达式自定义日志格式，参考：正则表达式教程

test.log

DEBUG:root:调试信息
INFO:root:普通信息
WARNING:root:警告信息
ERROR:root:错误信息
CRITICAL:root:严重错误信息
2019-12-15 20:17:02 - MainThread - root - DEBUG - DEBUG.
2019-12-15 20:17:02 - MainThread - root - INFO - INFO.
2019-12-15 20:17:02 - MainThread - root - WARNING - WARNING.
2019-12-15 20:17:02 - MainThread - root - ERROR - ERROR.
2019-12-15 20:17:02 - MainThread - root - CRITICAL - CRITICAL.

设置格式

Pattern	Action
\s*WARNING\s*	Highlight line
\s*ERROR\s*	Highlight line+stripe
\s*CRITICAL\s*	Highlight line+stripe
\s*DEBUG\s*	Highlight line

显示效果

封装

参考 Python常用库 - logging日志库 的封装，不是很好用，无法显示是哪个文件调用的，所以也找不到错误的源头

使用方法：

from log import logger

logger.debug('调试消息')
logger.info('普通消息')
logger.warn('警告消息')
logger.error('错误消息')
logger.critical('严重错误消息')

log.py

import logging
from logging import handlers


class Logger:
    __instance = None

    def __new__(cls, *args, **kwargs):
        if not cls.__instance:
            cls.__instance = object.__new__(cls, *args, **kwargs)
        return cls.__instance

    def __init__(self):
        # formater = logging.Formatter('%(message)s')  # 只要原本信息
        formater = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s', datefmt='%Y-%m-%d %H:%M:%S')  # 设置输出格式
        self.logger = logging.getLogger('log')  # 定义一个日志收集器
        self.logger.setLevel(logging.INFO)  # 设定级别
        self.fileLogger = handlers.RotatingFileHandler("./test.log", maxBytes=5242880, backupCount=3,
                                                       encoding='utf-8')  # 输出渠道一 - 文件形式
        self.console = logging.StreamHandler()  # 输出渠道二 - 控制台
        self.console.setLevel(logging.INFO)  # 控制台输出级别
        self.console.setFormatter(formater)  # 输出渠道对接输出格式
        self.fileLogger.setFormatter(formater)
        self.logger.addHandler(self.fileLogger)  # 日志收集器对接输出渠道
        self.logger.addHandler(self.console)

    def debug(self, msg):
        self.logger.debug(msg=msg)

    def info(self, msg):
        self.logger.info(msg=msg)

    def warn(self, msg):
        self.logger.warning(msg=msg)

    def error(self, msg):
        self.logger.error(msg=msg)

    def critical(self, msg):
        self.logger.critical(msg=msg)

    def excepiton(self, msg):
        self.logger.exception(msg=msg)


logger = Logger()

if __name__ == '__main__':
    logger.debug('调试消息')
    logger.info('普通消息')
    logger.warn('警告消息')
    logger.error('错误消息')
    logger.critical('严重错误消息')
    try:
        1 / 0
    except Exception as e:
        logger.excepiton(e)

loguru入门

安装

pip install loguru

无需样板即可使用

from loguru import logger

logger.debug('调试消息')
logger.info('普通消息')
logger.warning('警告消息')
logger.error('错误消息')
logger.critical('严重错误消息')

推荐阅读：Python日志库loguru——轻松记日志，一个函数搞定

Eliot入门

安装

pip install eliot eliot-tree

import requests
from eliot import start_action, to_file
to_file(open("linkcheck.log", "w"))


def check_links(urls):
    with start_action(action_type="check_links", urls=urls):
        for url in urls:
            try:
                with start_action(action_type="download", url=url):
                    response = requests.get(url)
                    response.raise_for_status()
            except Exception as e:
                raise ValueError(str(e))

try:
    check_links(["http://eliot.readthedocs.io", "http://nosuchurl"])
except ValueError:
    print("Not all links were valid.")

linkcheck.log

{"urls": ["http://eliot.readthedocs.io", "http://nosuchurl"], "action_status": "started", "timestamp": 1595411823.472086, "task_uuid": "532571b5-59a1-4fe8-983b-ed35101c9dfc", "action_type": "check_links", "task_level": [1]}
{"url": "http://eliot.readthedocs.io", "action_status": "started", "timestamp": 1595411823.472086, "task_uuid": "532571b5-59a1-4fe8-983b-ed35101c9dfc", "action_type": "download", "task_level": [2, 1]}
{"action_status": "succeeded", "timestamp": 1595411823.726474, "task_uuid": "532571b5-59a1-4fe8-983b-ed35101c9dfc", "action_type": "download", "task_level": [2, 2]}
{"url": "http://nosuchurl", "action_status": "started", "timestamp": 1595411823.726474, "task_uuid": "532571b5-59a1-4fe8-983b-ed35101c9dfc", "action_type": "download", "task_level": [3, 1]}
{"errno": null, "exception": "requests.exceptions.ConnectionError", "reason": "HTTPConnectionPool(host='nosuchurl', port=80): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x000000000B398320>: Failed to establish a new connection: [Errno 11004] getaddrinfo failed',))", "action_status": "failed", "timestamp": 1595411826.2962916, "task_uuid": "532571b5-59a1-4fe8-983b-ed35101c9dfc", "action_type": "download", "task_level": [3, 2]}
{"exception": "builtins.ValueError", "reason": "HTTPConnectionPool(host='nosuchurl', port=80): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x000000000B398320>: Failed to establish a new connection: [Errno 11004] getaddrinfo failed',))", "action_status": "failed", "timestamp": 1595411826.2962916, "task_uuid": "532571b5-59a1-4fe8-983b-ed35101c9dfc", "action_type": "check_links", "task_level": [4]}

树状可视化

eliot-tree linkcheck.log

效果

b1cb58cf-2c2f-45c0-92b2-838ac00b20cc
└── check_links/1 ⇒ started
    ├── timestamp: 2017-10-27 20:42:47.206684
    ├── urls:
    │   ├── 0: http://eliot.readthedocs.io
    │   └── 1: http://nosuchurl
    ├── download/2/1 ⇒ started
    │   ├── timestamp: 2017-10-27 20:42:47.206933
    │   ├── url: http://eliot.readthedocs.io
    │   └── download/2/2 ⇒ succeeded
    │       └── timestamp: 2017-10-27 20:42:47.439203
    ├── download/3/1 ⇒ started
    │   ├── timestamp: 2017-10-27 20:42:47.439412
    │   ├── url: http://nosuchurl
    │   └── download/3/2 ⇒ failed
    │       ├── errno: None
    │       ├── exception: requests.exceptions.ConnectionError
    │       ├── reason: HTTPConnectionPool(host='nosuchurl', port=80): Max retries exceeded with url: / (Caused by NewConnec…
    │       └── timestamp: 2017-10-27 20:42:47.457133
    └── check_links/4 ⇒ failed
        ├── exception: builtins.ValueError
        ├── reason: HTTPConnectionPool(host='nosuchurl', port=80): Max retries exceeded with url: / (Caused by NewConnec…
        └── timestamp: 2017-10-27 20:42:47.457332

参考文献

1. logging — Python 的日志记录工具
2. 哪些 Python 库让你相见恨晚？
3. itamarst/eliot: Eliot: the logging system that tells you why it happened
4. Eliot Documentation
5. Python常用库 - logging日志库
6. logging基础教程
7. logging进阶教程
8. Delgan/loguru: Python logging made (stupidly) simple
9. loguru documentation