Python进阶篇--logging模块的使用(下)

上一篇写了一些关于logging模块的基本使用方法，今天我们来一起挖掘一下，logging模块的高端用法。
下面是很好的一些参考资源，大家也可一拜读原文。
logging模块基础和进阶
logging handlers
logging cookbook

logging Filter

import logging
import logging.config
import time
import os


class CustomFilter(logging.Filter):

    def filter(self, record) -> int:
        if "example" in record.msg:
            record.msg = "This is msg is filtered and changed"
        return True

def log_module():

    # config log
    logging.basicConfig(
        level=logging.INFO
    )
    
    logger = logging.getLogger("e.g.")
    logger.addFilter(CustomFilter())
    logger.info("example: this is an example")
    logger.error("An error happened")


if __name__ == "__main__":
    log_module()

输出结果如下：

INFO:e.g.:This is msg is filtered and changed
ERROR:e.g.:An error happened

根据输出可以看的出来，这个含有example的log信息已经给修改成filter中指定的信息。至于如何进行过滤这个需要大家根据实际的需求来解决了。

logging Handler
对于handler这一块，python提供了非常非常丰富的接口。下面我们来一个一个看一下。
logging.StreamHandler 和logging.FileHandler，这两个非常相似，File Handler继承于StreamHandler. 用法比较简单我们只举一个简单的例子。

import logging
import logging.config
import time
import os


class CustomFilter(logging.Filter):

    def filter(self, record) -> int:
        if "example" in record.msg:
            record.msg = "This is msg is filtered and changed"
        return True


def log_module():

    # config log
    logging.basicConfig(
        level=logging.INFO
    )
    with open("app.log", "w") as f:
        stream_handler = logging.StreamHandler(stream=f)
        logger = logging.getLogger("e.g.")
        logger.addFilter(CustomFilter())
        logger.addHandler(stream_handler)
        logger.info("example: this is an example")
        logger.error("An error happened")


if __name__ == "__main__":
    log_module()

上面代码运行结果如下：
在终端上会输出如下信息：

INFO:e.g.:This is msg is filtered and changed
ERROR:e.g.:An error happened

在app.log中会输出如下信息：

This is msg is filtered and changed
An error happened

logging.NullHandler
The NullHandler class, located in the core logging package, does not do any formatting or output. It is essentially a ‘no-op’ handler for use by library developers. 这个就是不做任何动作，没有任何的输出。

logging.WatchedFileHandler
The WatchedFileHandler class, located in the logging.handlers module, is a FileHandler which watches the file it is logging to. If the file changes, it is closed and reopened using the file name.这个handler主要用在unix和linux系统上，windows上不能用。因为在windows系统下，正在被打开的文件是不会被改变的。

logging.handlers.RotatingFileHandler
RotatingFileHandler的参数比较复杂。
class logging.handlers.RotatingFileHandler(filename, mode=‘a’, maxBytes=0, backupCount=0, encoding=None, delay=False)
filename:handler要输出的目标文件
mode:hanlder要输出的目标文件的模式
maxBytes:一个文件的最大字节数当大于这个自己数的时候，就会新建一个文件。 比如： app.log，app.log.1, app.log.2 …。
backupCount:这个是生成新文件的最大数目， app.log这个文件本身除外。
delay:If delay is true, then file opening is deferred until the first call to emit(). emit函数的作用就是把内容输出到文件。

import logging
import logging.handlers
import logging.config
import time
import os


def log_module():
    # config log
    logging.basicConfig(
        level=logging.INFO,
        format='%(asctime)s %(name)-12s %(levelname)-8s %(message)s'
    )

    rotate_file_handler = logging.handlers.RotatingFileHandler("app.log", maxBytes=200, backupCount=10)
    formatter = logging.Formatter('%(asctime)s %(name)-12s %(levelname)-8s %(message)s')
    rotate_file_handler.setFormatter(formatter)
    logger = logging.getLogger("e.g.")
    logger.addHandler(rotate_file_handler)

    while True:
        logger.info("example: this is an example")
        logger.error("An error happened")


if __name__ == "__main__":
    log_module()

上述代码的输出结果：
终端输出如下：

2020-05-30 23:34:14,748 e.g. ERROR An error happened
2020-05-30 23:34:14,748 e.g. INFO example: this is an example
2020-05-30 23:34:14,755 e.g. ERROR An error happened
2020-05-30 23:34:14,755 e.g. INFO example: this is an example
2020-05-30 23:34:14,763 e.g. ERROR An error happened

同时会有11个文件生成：
app.log, app.log.1, app.log.2, app.log.3, …, app.log.10

logging.Handler.TimedFileHandler
这一个和上边的RotatingFileHandler功能很类似，上一个是依靠字节数来实现的，这一个是依靠时间来实现的，这里就不多赘述了。

logging.Hanlder.SocketHandler
有了这个Handler我们就可以轻松实现远程打印log的功能，如果觉得功能不够强大我们可以自己设计一个基于网络的log打印工具。

客户端打印log的代码如下

import logging
import logging.handlers


def socket_log():
    root_logger = logging.getLogger('')
    root_logger.setLevel(logging.DEBUG)
    socket_handler = logging.handlers.SocketHandler('localhost',
                                                    logging.handlers.DEFAULT_TCP_LOGGING_PORT)
    # don't bother with a formatter, since a socket handler sends the event as
    # an unformatted pickle
    root_logger.addHandler(socket_handler)

    # Now, we can log to the root logger, or any other logger. First the root...
    logging.info('Jackdaws love my big sphinx of quartz.')

    # Now, define a couple of other loggers which might represent areas in your
    # application:

    logger1 = logging.getLogger('myapp.area1')
    logger2 = logging.getLogger('myapp.area2')

    logger1.debug('Quick zephyrs blow, vexing daft Jim.')
    logger1.info('How quickly daft jumping zebras vex.')
    logger2.warning('Jail zesty vixen who grabbed pay from quack.')
    logger2.error('The five boxing wizards jump quickly.')


if __name__ == "__main__":
    socket_log()

server端的代码如下

import pickle
import logging
import logging.handlers
import socketserver
import struct


class LogRecordStreamHandler(socketserver.StreamRequestHandler):
    """Handler for a streaming logging request.

    This basically logs the record using whatever logging policy is
    configured locally.
    """

    def handle(self):
        """
        Handle multiple requests - each expected to be a 4-byte length,
        followed by the LogRecord in pickle format. Logs the record
        according to whatever policy is configured locally.
        """
        while True:
            chunk = self.connection.recv(4)
            if len(chunk) < 4:
                break
            slen = struct.unpack('>L', chunk)[0]
            chunk = self.connection.recv(slen)
            while len(chunk) < slen:
                chunk = chunk + self.connection.recv(slen - len(chunk))
            obj = self.unPickle(chunk)
            record = logging.makeLogRecord(obj)
            self.handleLogRecord(record)

    def unPickle(self, data):
        return pickle.loads(data)

    def handleLogRecord(self, record):
        # if a name is specified, we use the named logger rather than the one
        # implied by the record.
        if self.server.logname is not None:
            name = self.server.logname
        else:
            name = record.name
        logger = logging.getLogger(name)
        # N.B. EVERY record gets logged. This is because Logger.handle
        # is normally called AFTER logger-level filtering. If you want
        # to do filtering, do it at the client end to save wasting
        # cycles and network bandwidth!
        logger.handle(record)


class LogRecordSocketReceiver(socketserver.ThreadingTCPServer):
    """
    Simple TCP socket-based logging receiver suitable for testing.
    """

    allow_reuse_address = True

    def __init__(self, host='localhost',
                 port=logging.handlers.DEFAULT_TCP_LOGGING_PORT,
                 handler=LogRecordStreamHandler):
        socketserver.ThreadingTCPServer.__init__(self, (host, port), handler)
        self.abort = 0
        self.timeout = 1
        self.logname = None

    def serve_until_stopped(self):
        import select
        abort = 0
        while not abort:
            rd, wr, ex = select.select([self.socket.fileno()],
                                       [], [],
                                       self.timeout)
            if rd:
                self.handle_request()
            abort = self.abort
            print("The abort is ", abort)


def main():
    logging.basicConfig(
        format='%(relativeCreated)5d %(name)-15s %(levelname)-8s %(message)s')
    tcpserver = LogRecordSocketReceiver()
    print('About to start TCP server...')
    tcpserver.serve_until_stopped()


if __name__ == '__main__':
    main()

server端的输出结果如下：

16 root INFO Jackdaws love my big sphinx of quartz.
1037 myapp.area1 DEBUG Quick zephyrs blow, vexing daft Jim.
1037 myapp.area1 INFO How quickly daft jumping zebras vex.
1037 myapp.area2 WARNING Jail zesty vixen who grabbed pay from quack.
1037 myapp.area2 ERROR The five boxing wizards jump quickly.

logging模块高端用法，就介绍到这里了，还有一些用法，大家可以到python的官网去自行探索。