logging多进程报错:PermissionError: [WinError 32] 另一个程序正在使用此文件,进程无法访问
问题重现
2019-12-23 13:28:24,415 - INFO: 正在进行D:\Anaconda_program\hangtian_spider\hangkong/data/EI/01.txt文件<RECORD 216>记录的读取
--- Logging error ---
Traceback (most recent call last):
File "D:\Anaconda3\lib\logging\handlers.py", line 72, in emit
self.doRollover()
File "D:\Anaconda3\lib\logging\handlers.py", line 173, in doRollover
self.rotate(self.baseFilename, dfn)
File "D:\Anaconda3\lib\logging\handlers.py", line 113, in rotate
os.rename(source, dest)
PermissionError: [WinError 32] 另一个程序正在使用此文件,进程无法访问。: 'D:\\Anaconda_program\\hangtian_spider\\hangkong\\log\\ei_txt_file.log' -> 'D:\\Anaconda_program\\hangtian_spider\\hangkong\\log\\ei_txt_file.log.1'
Call stack:
File "D:\Anaconda3\lib\threading.py", line 884, in _bootstrap
self._bootstrap_inner()
File "D:\Anaconda3\lib\threading.py", line 916, in _bootstrap_inner
self.run()
File "D:\Anaconda3\lib\threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "D:\Anaconda_program\hangtian_spider\hangkong\web\app\data\EI_Data.py", line 228, in get_ei_txt_file
read_txt(root_dir+file)
File "D:\Anaconda_program\hangtian_spider\hangkong\web\app\data\EI_Data.py", line 260, in read_txt
ei_data.add_ei_data(re_dict)
File "D:\Anaconda_program\hangtian_spider\hangkong\web\app\data\EI_Data.py", line 126, in add_ei_data
logger.info("本条记录中筛选的关键字段有:record_dict.keys(): {}".format(', '.join(record_dict.keys())))
Message: '本条记录中筛选的关键字段有:record_dict.keys(): ei_an, ei_ti, ei_au, ei_af, ei_ca, ei_st, ei_ast, ei_vl, ei_is, ei_isd, ei_py, ei_pg, ei_la, ei_sn, ei_cn, ei_dt, ei_pu, ei_ab, ei_nr, ei_mh, ei_ctt, ei_uctt, ei_cfc, ei_di, ei_db, ei_cit'
Arguments: ()
2019-12-23 13:28:24,467 - INFO: 本条记录中筛选的关键字段有:record_dict.keys(): ei_an, ei_ti, ei_au, ei_af, ei_ca, ei_st, ei_ast, ei_vl, ei_is, ei_isd, ei_py, ei_pg, ei_la, ei_sn, ei_cn, ei_dt, ei_pu, ei_ab, ei_nr, ei_mh, ei_ctt, ei_uctt, ei_cfc, ei_di, ei_db, ei_cit
--- Logging error ---
解决办法
from logging import getLogger, INFO
from concurrent_log_handler import ConcurrentRotatingFileHandler
import os
log = getLogger()
# Use an absolute path to prevent file rotation trouble.
logfile = os.path.abspath("mylogfile.log")
# Rotate log after reaching 512K, keep 5 old copies.
rotateHandler = ConcurrentRotatingFileHandler(logfile, "a", 512*1024, 5)
log.addHandler(rotateHandler)
log.setLevel(INFO)
log.info("Here is a very exciting log message, just for you")
参考
https://github.com/Preston-Landers/concurrent-log-handler
https://blog.csdn.net/chongtong/article/details/80831782