问题:
[scrapy] ERROR: Error downloading
GET http://www.dmoz.org/robots.txt: ‘float’ object is not iterable
Traceback (most recent call last):
File “d:\Anaconda3\lib\site-packages\twisted\internet\defer.py”, line 1299, in _inlineCallbacks
result = result.throwExceptionIntoGenerator(g)
File “d:\Anaconda3\lib\site-packages\twisted\python\failure.py”, line 393, in throwExceptionIntoGenerator
return g.throw(self.type, self.value, self.tb)
File “d:\Anaconda3\lib\site-packages\scrapy\core\downloader\middleware.py”, line 43, in process_request
defer.returnValue((yield download_func(request=request,spider=spider)))
File “d:\Anaconda3\lib\site-packages\scrapy\utils\defer.py”, line 45, in mustbe_deferred
result = f(*args, **kw)
…
scrapy与twisted版本不兼容。见https://github.com/scrapy/scrapy/issues/2606