python scrapy爬虫遇见301_#0 scrapy爬虫学习中遇到的坑记录

2018-07-26 09:37:28 [scrapy.utils.log] INFO: Scrapy 1.5.0 started (bot: jizhi)2018-07-26 09:37:28 [scrapy.utils.log] INFO: Versions: lxml 4.2.1.0, libxml2 2.9.8, cssselect 1.0.3, parsel 1.4.0, w3lib 1.19.0, Twisted 17.5.0, Python 3.6.5 |Anaconda, Inc.| (default, Mar 29 2018, 13:32:41) [MSC v.1900 64 bit (AMD64)], pyOpenSSL 18.0.0 (OpenSSL 1.0.2k 26 Jan 2017), cryptography 2.2.2, Platform Windows-10-10.0.17134-SP02018-07-26 09:37:28 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'jizhi', 'NEWSPIDER_MODULE': 'jizhi.spiders', 'SPIDER_MODULES': ['jizhi.spiders']}2018-07-26 09:37:29[scrapy.middleware] INFO: Enabled extensions:

['scrapy.extensions.corestats.CoreStats','scrapy.extensions.telnet.TelnetConsole','scrapy.extensions.logstats.LogStats']2018-07-26 09:37:29[scrapy.middleware] INFO: Enabled downloader middlewares:

['scrapy.downloadermiddlewares.htt