[scrapy] DEBUG: Telnet console listening on 127.0.0.1:6023”
“ DEBUG: Crawled (403) <GET
403是禁止访问,查到
代理问题导致有些网站无法访问,解决方法是在settings.py中修改:
DOWNLOADER_MIDDLEWARES = {
'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware': None,
}