python爬虫运行正常最后报错_python爬虫运行过程报错,求救

d:\Python\ajk>scrapy crawl fangjia

2017-06-08 20:17:26 [scrapy] INFO: Scrapy 1.0.3 started (bot: ajk)

2017-06-08 20:17:26 [scrapy] INFO: Optional features available: ssl, http11

2017-06-08 20:17:26 [scrapy] INFO: Overridden settings: {'NEWSPIDER_MODULE': 'ajk.spiders', 'SPIDER_MODULES': ['ajk.spiders'], 'BOT_NAME': 'ajk'}

2017-06-08 20:17:26 [scrapy] INFO: Enabled extensions: CloseSpider, TelnetConsole, LogStats, CoreStats, SpiderState

2017-06-08 20:17:27 [scrapy] INFO: Enabled downloader middlewares: HttpAuthMiddleware, DownloadTimeoutMiddleware, UserAgentMiddleware, RetryMiddleware, DefaultHeadersMiddleware, MetaRefreshMiddleware, HttpCompressionMiddleware, RedirectMiddleware, CookiesMiddleware, ChunkedTransferMiddleware, DownloaderStats

2017-06-08 20:17:27 [scrapy] INFO: Enabled spider middlewares: HttpErrorMiddleware, OffsiteMiddleware, RefererMiddleware, UrlLengthMiddleware, DepthMiddleware

2017-06-08 20:17:27 [scrapy] INFO: Enabled item pipelines: AjkPipeline

2017-06-08 20:17:27 [scrapy] INFO: Spider opened

2017-06-08 20:17:27 [scrapy] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)

2017-06-08 20:17:27 [scrapy] DEBUG: Telnet console listening on 127.0.0.1:6023

2017-06-08 20:17:27 [scrapy] DEBUG: Redirecting (301) to from [/url]

2017-06-08 20:17:27 [scrapy] DEBUG: Redirecting (302) to from

2017-06-08 20:17:27 [scrapy] DEBUG: Crawled (404) (referer: None)

2017-06-08 20:17:27 [scrapy] DEBUG: Ignoring response <404 [url]http://hangzhou.anjuke.com/404/?from=antispam>: HTTP status code is not handled or not allowed

2017-06-08 20:17:27 [scrapy] INFO: Closing spider (finished)

2017-06-08 20:17:27 [scrapy] INFO: Dumping Scrapy stats:

{'downloader/request_bytes': 856,

'downloader/request_count': 3,

'downloader/request_method_count/GET': 3,

'downloader/response_bytes': 10394,

'downloader/response_count': 3,

'downloader/response_status_count/301': 1,

'downloader/response_status_count/302': 1,

'downloader/response_status_count/404': 1,

'finish_reason': 'finished',

'finish_time': datetime.datetime(2017, 6, 8, 12, 17, 27, 808000),

'log_count/DEBUG': 5,

'log_count/INFO': 7,

'response_received_count': 1,

'scheduler/dequeued': 3,

'scheduler/dequeued/memory': 3,

'scheduler/enqueued': 3,

'scheduler/enqueued/memory': 3,

'start_time': datetime.datetime(2017, 6, 8, 12, 17, 27, 415000)}

2017-06-08 20:17:27 [scrapy] INFO: Spider closed (finished)

  • 0
    点赞
  • 0
    收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
©️2022 CSDN 皮肤主题:数字20 设计师:CSDN官方博客 返回首页
评论
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、C币套餐、付费专栏及课程。

余额充值