python 安装scrapy模块

1: pip install wheel
2: pip install lxml
3: pip install PyOpenssl

4. 安装scrapy模块需要安装安装安装 microsoft visual c++ build tools（直接安装个Visual Studio)

即可。

5.安装twisted模块（pip install twisted）,可能出错

Running setup.py bdist_wheel for Twisted … error

Failed building wheel for Twisted

Running setup.py clean for Twisted

Failed to build Twisted

Installing collected packages: pycparser, cffi, asn1crypto, cryptography, pyOpenSSL, Twisted, pyasn1, pyasn1-modules, service-identity, queuelib, scrapy

Running setup.py install for Twisted … error....

 修改该文件第75行的“UTF-8”为”mbcs", 安装成功scrapy,改回UTF-8.

 6 .安装twisted模块成功之后，直接安装scrapy模块（pip install scapy）.

测试是否安装成功

在命令行中，首先创建项目
scrapy startproject hello
进入项目目录
cd hello
生成爬虫
scrapy genspider baidu www.baidu.com
爬取百度
scrapy crawl baidu

如果命令都可以正常运行，则证明安装成功。
参考：https://blog.csdn.net/qq_29186489/article/details/78736945