替换文件中的敏感信息python_屏蔽python日志中的敏感信息

Consider the following code

try:

r = requests.get('https://sensitive:passw0rd@what.ever/')

r.raise_for_status()

except requests.HTTPError:

logging.exception("Failed to what.ever")

Here, if the endpoint returns non-successful http status code, the following will be logged

Traceback (most recent call last):

File "a.py", line 5, in

r.raise_for_status()

File "venv/lib/python3.5/site-packages/requests/models.py", line 928, in raise_for_status

raise HTTPError(http_error_msg, response=self)

requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://sensitive:passw0rd@what.ever/

The problem is that the password is logged. I could create a logging filter to filter out this line completely. However, it would be more convenient if the password was just masked out somehow. As no string is passed to logging.exception filtering on the app side is tricky. Where in the logging framwork can I transform a log record?

解决方案

Apparently, this is done with a Formatter. Example below

import logging

import re

class SensitiveFormatter(logging.Formatter):

"""Formatter that removes sensitive information in urls."""

@staticmethod

def _filter(s):

return re.sub(r':\/\/(.*?)\@', r'://', s)

def format(self, record):

original = logging.Formatter.format(self, record)

return self._filter(original)

Use like so

import logging

import requests

from sensitive_formatter import SensitiveFormatter

LOG_FORMAT = \

'%(asctime)s [%(threadName)-16s] %(filename)27s:%(lineno)-4d %(levelname)7s| %(message)s'

logging.basicConfig(level=logging.DEBUG)

log = logging.getLogger(__name__)

# Don't actually configure your logging like this, just to showcase

# the above answer. :)

for handler in logging.root.handlers:

handler.setFormatter(SensitiveFormatter(LOG_FORMAT))

log.warning('https://not:shown@httpbin.org/basic-auth/expected-user/expected-pass')

try:

r = requests.get('https://not:shown@httpbin.org/basic-auth/expected-user/expected-pass')

r.raise_for_status()

except requests.exceptions.RequestException as e:

log.exception('boom!')

The user/password will be masked out. See example log below

$ python log_example.py

2018-05-18 11:59:22,703 [MainThread ] log.py:14 WARNING| https://httpbin.org/basic-auth/user/secret

2018-05-18 11:59:22,747 [MainThread ] connectionpool.py:824 DEBUG| Starting new HTTPS connection (1): httpbin.org

2018-05-18 11:59:23,908 [MainThread ] connectionpool.py:396 DEBUG| https://httpbin.org:443 "DELETE /basic-auth/user/secret HTTP/1.1" 405 178

2018-05-18 11:59:23,913 [MainThread ] log.py:19 ERROR| boom!

Traceback (most recent call last):

File "log.py", line 17, in

r.raise_for_status()

File "/Users/vidstige/src/so/venv/lib/python3.6/site-packages/requests/models.py", line 935, in raise_for_status

raise HTTPError(http_error_msg, response=self)

requests.exceptions.HTTPError: 405 Client Error: METHOD NOT ALLOWED for url: https://httpbin.org/basic-auth/user/secret