【Scrapy】Scrapy在Python3下报错：“cannot import name '_win32stdio'”解决办法

由于之前都是在Python2的环境下用的Scrapy，然后最近在用Python3的环境，今天试着用Scrapy启动一个爬虫的时候居然报错。

• 操作

scrapy crawl demo

• 报错信息

Traceback (most recent call last):
  File "c:\python34\lib\site-packages\scrapy\commands\crawl.py", line 57, in run
    self.crawler_process.crawl(spname, **opts.spargs)
  File "c:\python34\lib\site-packages\scrapy\crawler.py", line 168, in crawl
    return self._crawl(crawler, *args, **kwargs)
  File "c:\python34\lib\site-packages\scrapy\crawler.py", line 172, in _crawl
    d = crawler.crawl(*args, **kwargs)
  File "c:\python34\lib\site-packages\twisted\internet\defer.py", line 1274, in unwindGenerator
    return _inlineCallbacks(None, gen, Deferred())
--- <exception caught here> ---
  File "c:\python34\lib\site-packages\twisted\internet\defer.py", line 1128, in _inlineCallbacks
    result = g.send(result)
  File "c:\python34\lib\site-packages\scrapy\crawler.py", line 77, in crawl
    self.engine = self._create_engine()
  File "c:\python34\lib\site-packages\scrapy\crawler.py", line 102, in _create_engine
    return ExecutionEngine(self, lambda _: self.stop())
  File "c:\python34\lib\site-packages\scrapy\core\engine.py", line 69, in __init__
    self.downloader = downloader_cls(crawler)
  File "c:\python34\lib\site-packages\scrapy\core\downloader\__init__.py", line 88, in __init__
    self.middleware = DownloaderMiddlewareManager.from_crawler(crawler)
  File "c:\python34\lib\site-packages\scrapy\middleware.py", line 58, in from_crawler
    return cls.from_settings(crawler.settings, crawler)
  File "c:\python34\lib\site-packages\scrapy\middleware.py", line 34, in from_settings
    mwcls = load_object(clspath)
  File "c:\python34\lib\site-packages\scrapy\utils\misc.py", line 44, in load_object
    mod = import_module(module)
  File "c:\python34\lib\importlib\__init__.py", line 109, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 2254, in _gcd_import

  File "<frozen importlib._bootstrap>", line 2237, in _find_and_load

  File "<frozen importlib._bootstrap>", line 2226, in _find_and_load_unlocked

  File "<frozen importlib._bootstrap>", line 1200, in _load_unlocked

  File "<frozen importlib._bootstrap>", line 1129, in _exec

  File "<frozen importlib._bootstrap>", line 1471, in exec_module

  File "<frozen importlib._bootstrap>", line 321, in _call_with_frames_removed

  File "c:\python34\lib\site-packages\scrapy\downloadermiddlewares\retry.py", line 20, in <module>
    from twisted.web.client import ResponseFailed
  File "c:\python34\lib\site-packages\twisted\web\client.py", line 41, in <module>
    from twisted.internet.endpoints import TCP4ClientEndpoint, SSL4ClientEndpoint
  File "c:\python34\lib\site-packages\twisted\internet\endpoints.py", line 34, in <module>
    from twisted.internet.stdio import StandardIO, PipeAddress
  File "c:\python34\lib\site-packages\twisted\internet\stdio.py", line 30, in <module>
    from twisted.internet import _win32stdio
builtins.ImportError: cannot import name '_win32stdio'
2017-10-19 09:35:53 [twisted] CRITICAL:
Traceback (most recent call last):
  File "c:\python34\lib\site-packages\twisted\internet\defer.py", line 1128, in _inlineCallbacks
    result = g.send(result)
  File "c:\python34\lib\site-packages\scrapy\crawler.py", line 77, in crawl
    self.engine = self._create_engine()
  File "c:\python34\lib\site-packages\scrapy\crawler.py", line 102, in _create_engine
    return ExecutionEngine(self, lambda _: self.stop())
  File "c:\python34\lib\site-packages\scrapy\core\engine.py", line 69, in __init__
    self.downloader = downloader_cls(crawler)
  File "c:\python34\lib\site-packages\scrapy\core\downloader\__init__.py", line 88, in __init__
    self.middleware = DownloaderMiddlewareManager.from_crawler(crawler)
  File "c:\python34\lib\site-packages\scrapy\middleware.py", line 58, in from_crawler
    return cls.from_settings(crawler.settings, crawler)
  File "c:\python34\lib\site-packages\scrapy\middleware.py", line 34, in from_settings
    mwcls = load_object(clspath)
  File "c:\python34\lib\site-packages\scrapy\utils\misc.py", line 44, in load_object
    mod = import_module(module)
  File "c:\python34\lib\importlib\__init__.py", line 109, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 2254, in _gcd_import
  File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
  File "<frozen importlib._bootstrap>", line 2226, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1200, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1129, in _exec
  File "<frozen importlib._bootstrap>", line 1471, in exec_module
  File "<frozen importlib._bootstrap>", line 321, in _call_with_frames_removed
  File "c:\python34\lib\site-packages\scrapy\downloadermiddlewares\retry.py", line 20, in <module>
    from twisted.web.client import ResponseFailed
  File "c:\python34\lib\site-packages\twisted\web\client.py", line 41, in <module>
    from twisted.internet.endpoints import TCP4ClientEndpoint, SSL4ClientEndpoint
  File "c:\python34\lib\site-packages\twisted\internet\endpoints.py", line 34, in <module>
    from twisted.internet.stdio import StandardIO, PipeAddress
  File "c:\python34\lib\site-packages\twisted\internet\stdio.py", line 30, in <module>
    from twisted.internet import _win32stdio
ImportError: cannot import name '_win32stdio'

• 抓重点

ImportError: cannot import name '_win32stdio'

• 查资料
  1.ImportError : cannot import name ‘_win32stdio’
  2.twisted on github

• 解决方案
  通过查找资料，发现twisted.internet._win32stdio does not work on Windows with Python 3.
  不过我们可以
  Copy the _win32stdio and _pollingfile to the appropriate directory under site-packages. Namely, twisted-dir\internet.
  即,将资料twisted on github中的_win32stdio.py和_pollingfile.py手动拷贝到Python3的Lib–>site-packages–>twisted–>internet目录下。

• 结果
  重新启动爬虫，顺利解决。