win10下基于python3.5安装scrapy教程

1、安装pywin32

下载文件  http://www.lfd.uci.edu/~gohlke/pythonlibs/#pywin32 ， 通过pip安装

pip install pywin32-220.1-cp35-cp35m-win32.whl

2、安装 twisted

下载文件 http://www.lfd.uci.edu/~gohlke/pythonlibs/#twisted  通过pip安装

pip install Twisted-16.5.0-cp35-cp35m-win32.whl

3、安装 scrapy

pip install scrapy

在这一步的安装中出现以下报错：

Command "c:\users\springlustre\appdata\local\programs\python\python35-32\python.exe -u -c "import setuptools, tokenize;_
_file__='C:\\Users\\SPRING~1\\AppData\\Local\\Temp\\pip-build-u69illst\\lxml\\setup.py';f=getattr(tokenize, 'open', open
)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record C:\Use
rs\SPRING~1\AppData\Local\Temp\pip-serz1p13-record\install-record.txt --single-version-externally-managed --compile" fai
led with error code 1 in C:\Users\SPRING~1\AppData\Local\Temp\pip-build-u69illst\lxml\

原因是在windows环境中，调用的vcvarsall.bat的文件在c++编程环境才会有。网上的解决方法有两个：一是安装MinGW；二是安装Visual Studio ，我用的是第二种

   下载安装visio studio 

https://www.visualstudio.com/zh-hans/post-download-vs/?sku=community&clcid=0x804&telem=ga

再次安装报错

 C:\Users\SPRING~1\AppData\Local\Temp\xmlXPathInit5xdue4a3.c(1): fatal error C1083: 无法打开包括文件: “libxml/xpath.
: No such file or directory
 *********************************************************************************
 Could not find function xmlCheckVersion in library libxml2. Is libxml2 installed?
 *********************************************************************************
 error: command 'E:\\Program Files\\Microsoft Visual Studio 14.0\\VC\\BIN\\cl.exe' failed with exit status 2

解决方法：安装lxml  http://www.lfd.uci.edu/~gohlke/pythonlibs/#lxml

运行命令  pip install lxml

运行的时候 报错

  

  File "c:\users\springlustre\appdata\local\programs\python\python35-32\lib\site-packages\scrapy\downloadermiddlewares\r
etry.py", line 23, in <module>
    from scrapy.xlib.tx import ResponseFailed
  File "c:\users\springlustre\appdata\local\programs\python\python35-32\lib\site-packages\scrapy\xlib\tx\__init__.py", l
ine 3, in <module>
    from twisted.web import client
  File "c:\users\springlustre\appdata\local\programs\python\python35-32\lib\site-packages\twisted\web\client.py", line 4
2, in <module>
    from twisted.internet.endpoints import TCP4ClientEndpoint, SSL4ClientEndpoint
  File "c:\users\springlustre\appdata\local\programs\python\python35-32\lib\site-packages\twisted\internet\endpoints.py"
, line 36, in <module>
    from twisted.internet.stdio import StandardIO, PipeAddress
  File "c:\users\springlustre\appdata\local\programs\python\python35-32\lib\site-packages\twisted\internet\stdio.py", li
ne 30, in <module>
    from twisted.internet import _win32stdio
  File "c:\users\springlustre\appdata\local\programs\python\python35-32\lib\site-packages\twisted\internet\_win32stdio.p
y", line 9, in <module>
    import win32api
ImportError: DLL load failed: 找不到指定的模块。

后来发现是win32的问题，现在已经改成pypiwin32了，

从https://pypi.python.org/pypi/pypiwin32 下载对应的版本，使用pip install安装接口