settings里面配置: DEFAULT_REQUEST_HEADERS={ "user-agent":"....", } middleware写: class HeadersMiddleware(object): def __init__(self,crawler): super(HeadersMiddleware,self).__init__() self.my_headers = crawler.settings.get("DEFAULT_REQUEST_HEADERS") @classmethod def from_crawler(cls,crawler): # 将它传递给self return cls(crawler) def process_request(self,request,spider): request.headers.setdefault('headers',self.my_headers)
scrapy downloadmiddleware的添加headersmiddleware
最新推荐文章于 2024-06-12 20:06:51 发布