scrapy downloadmiddleware的添加headersmiddleware
settings里面配置:
DEFAULT_REQUEST_HEADERS={
"user-agent":"....",
}
middleware写:
class HeadersMiddleware(object):
def __init__(self,crawler):
super(HeadersMiddleware,self).__init__()
s...
原创
2019-01-10 11:04:25 ·
647 阅读 ·
0 评论