scrapy.log模块已经被弃用
警告看着难受,可以直接使用Logger类,因为scrapy.log也就是封装了一层的Logger
示例:
import logging
logger = logging.getLogger(__name__)
logger.log(logging.WARNING, '警告') # 输出日志
Logger接口:
class Logger(Filterer):
def setLevel(self, level):
def debug(msg):
def info(msg):
def warning(msg):
def warn(msg):
def error(msg):
def excption(msg):
def critical(msg):
def log(level, msg):
...
还有一种用法,每个spider都有自己的logger,可以直接调用,比如在spider中这样用:
def parse(self, response):
self.logger.info(response.url)
在管道中:
def process_item(self, item, spider):
spider.logger.info(item)
return item
scrapy源码如下:
class Spider(object_ref):
"""Base class for scrapy spiders. All spiders must inherit from this
class.
"""
name = None
custom_settings = None
def __init__(self, name=None, **kwargs):
if name is not None:
self.name = name
elif not getattr(self, 'name', None):
raise ValueError("%s must have a name" % type(self).__name__)
self.__dict__.update(kwargs)
if not hasattr(self, 'start_urls'):
self.start_urls = []
@property
def logger(self):
logger = logging.getLogger(self.name)
return logging.LoggerAdapter(logger, {'spider': self})
def log(self, message, level=logging.DEBUG, **kw):
"""Log the given message at the given log level
This helper wraps a log call to the logger within the spider, but you
can use it directly (e.g. Spider.logger.info('msg')) or use any other
Python logger too.
"""
self.logger.log(level, message, **kw)