在windows10系统安装scrapy框架过程中遇到一些问题,以备后人少走弯路。
安装scrapy最主要的是需要安装它所依赖的安装包。
1、安装lxml, lxml是一种使用 Python 编写的库,可以迅速、灵活地处理 XML。
安装命令:python -m pip install lxml 直接在命令行cmd下运行。提示如下提示语即为成功Successfully installed lxml-4.1.1
2、安装setuptools, 一般都已经安装了的,可在cmd中用 python -m pip list 查看是否已经安装,如果没安装,用下载命令:python -m pip install setuptools安装。一般电脑这个都已经安装过了。
3、安装zope.interface,在cmd中用下载命令:python -m pip install zope.interface。提示如下提示语即为安装成功Successfully installed zope.interface-4.4.3
4、安装Twisted,在cmd中用下载命令:python -m pip install Twisted,它是用Python实现的基于事件驱动的网络引擎框架。提示如下提示语即为安装成功Successfully installed Automat-0.6.0 Twisted-17.9.0 attrs-17.4.0 constantly-15.1.0 hyperlink-17.3.1 incremental-17.5.0
5、安装pyOpenSSL,pyOpenSSL是Python的OpenSSL接口, 可在cmd中用下载命令:python -m pip install pyOpenSSL安装,提示 如下提示语即为安装成功Successfully installed asn1crypto-0.24.0 cffi-1.11.4 cryptography-2.1.4 enum34-1.1.6 idna-2.6 ipaddress-1.0.19 pycparser-2.18
6、安装win32py,可以下载我分享的资源,进行安装。http://download.csdn.net/download/zxcvbnm1234ws/10219726
7、安装Scrapy,直接在cmd中输入easy_install scrapy回车即可,如果上面6步都不出任何问题顺利安装完成的话,这一步应该会顺利安装完成,并且安装完成后需要在命令行下面输入scrapy,如果提示以下提示语,恭喜你你以成功安装Scrapy,你可以利用该框架实现网络爬虫了。
Scrapy 1.5.0 - no active project
Usage:
scrapy <command> [options] [args]
Available commands:
bench Run quick benchmark test
fetch Fetch a URL using the Scrapy downloader
genspider Generate new spider using pre-defined templates
runspider Run a self-contained spider (without creating a project)
settings Get settings values
shell Interactive scraping console
startproject Create new project
version Print Scrapy version
view Open URL in browser, as seen by Scrapy
[ more ] More commands available when run from project directory
Use "scrapy <command> -h" to see more info about a command
如果没有安装过程中报错,提示如下错误语:error: The 'pyasn1' distribution was not found and is required by service-identity 应该是之前安装scrapy没有安装成功导致的,需要更新一下scrapy,只需要在命令行下输入以下命令并运行:pip install --upgrade scrapy,运行完成提示如下提示语即为安装成功Installing collected packages: pyasn1, setuptools
Found existing installation: setuptools 28.8.0
Uninstalling setuptools-28.8.0:
Successfully uninstalled setuptools-28.8.0
Successfully installed pyasn1-0.4.2 setuptools-38.4.0