最近用datacompy来对比数据,好用是好用,但是总是有莫名的log error,如下:
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - --- Logging error ---
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - Traceback (most recent call last):
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - File "/miniconda/envs/database_env/lib/python3.8/logging/__init__.py", line 440, in format
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - return self._format(record)
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - File "/miniconda/envs/database_env/lib/python3.8/logging/__init__.py", line 436, in _format
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - return self._fmt % record.__dict__
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - KeyError: 'scriptname'
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO -
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - During handling of the above exception, another exception occurred:
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO -
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - Traceback (most recent call last):
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - File "/miniconda/envs/database_env/lib/python3.8/logging/__init__.py", line 1085, in emit
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - msg = self.format(record)
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - File "/miniconda/envs/database_env/lib/python3.8/logging/__init__.py", line 929, in format
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - return fmt.format(record)
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - File "/miniconda/envs/database_env/lib/python3.8/logging/__init__.py", line 671, in format
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - s = self.formatMessage(record)
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - File "/miniconda/envs/database_env/lib/python3.8/logging/__init__.py", line 640, in formatMessage
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - return self._style.format(record)
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - File "/miniconda/envs/database_env/lib/python3.8/logging/__init__.py", line 442, in format
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - raise ValueError('Formatting field not found in record: %s' % e)
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - ValueError: Formatting field not found in record: 'scriptname'
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - Call stack:
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - File "part1_stock_mark.py", line 471, in <module>
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - check_ms_ch(data)
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - File "part1_stock_mark.py", line 453, in check_ms_ch
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - if not datacompy.Compare(chdata, msdata, join_columns=['stock_code', 'trade_date']).matches():
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - File "/root/.local/lib/python3.8/site-packages/datacompy/core.py", line 124, in __init__
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - self._compare(ignore_spaces, ignore_case)
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - File "/root/.local/lib/python3.8/site-packages/datacompy/core.py", line 195, in _compare
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - LOG.info("df1 does not Pandas.DataFrame.equals df2")
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - Message: 'df1 does not Pandas.DataFrame.equals df2'
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - Arguments: ()
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - --- Logging error ---
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - Traceback (most recent call last):
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - File "/miniconda/envs/database_env/lib/python3.8/logging/__init__.py", line 440, in format
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - return self._format(record)
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - File "/miniconda/envs/database_env/lib/python3.8/logging/__init__.py", line 436, in _format
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - return self._fmt % record.__dict__
[2022-01-13, 18:05:05 CST] {subprocess.py:89} INFO - KeyError: 'scriptname'
就是说datacompy在使用logging.info时缺了一个scriptname的参数。这也不是必须的参数啊,我搜变logging的文档也没找到关于这个参数的信息,网上也没有相关的经验。
其实这个问题可以通过如下的方式解决,就是在调用info 的时候传入extra参数
logging.info(msg, extra={'scriptname':''})
但是这个info是datacompy调用的又不是我自己…
所以只能在每次导入datacompy的时候,设置日志安全等级,避开这个info提醒
import datacompy
datacompy.LOG.setLevel(logging.WARNING)
但我感觉这个方式一点也不优雅,要是哪位大神有更好的解决方案麻烦告诉我,谢谢