爬虫----090scrapy的安装

编程菜鸟Q

已于 2024-04-02 17:06:18 修改

阅读量414

点赞数 8

分类专栏：爬虫小白学习文章标签：爬虫前端 python 开发语言

于 2024-03-25 19:56:43 首次发布

本文链接：https://blog.csdn.net/m0_74151139/article/details/137023555

版权

爬虫小白学习专栏收录该内容

19 篇文章 0 订阅

订阅专栏

1、安装时选择国内源，国外源安装极慢

2、在我安装时出现了一些错误，在此分享一下

【注意】需要先下载scrapy的依赖库twisted，根据python版本选择对应的twisted库

地址：http://www.lfd.uci.edu/~gohlke/pythonlibs/#twisted

下载 twisted 对应版本的 whl文件（比如我的twisted_iocpsupport-1.0.2-cp38-cp38-win_amd64.whl）， cp 后面是 python版本，amd64 代表 64 位。

错误1 此处可选择更换下载源

我成功下载的源头为：

pip install scrapy -i http://pypi.doubanio.com/simple/ --trusted-host pypi.doubanio.com

#WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))': /simple/scrapy/
# WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))': /simple/scrapy/
# WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))': /simple/scrapy/
# WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))': /simple/scrapy/
# WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))': /simple/scrapy/
# ERROR: Could not find a version that satisfies the requirement scrapy (from versions: none)
# ERROR: No matching distribution found for scrapy

错误2 pip版本过低升级pip版本

           python ‐m pip install ‐‐upgrade pip

总步骤：

###在电脑终端里输入

python ‐m pip install ‐‐upgrade pip

pip D:\twisted\twisted_iocpsupport-1.0.2-cp38-cp38-win_amd64.whl    #此处写自己的库地址

pip install scrapy -i http://pypi.doubanio.com/simple/ --trusted-host pypi.doubanio.com

然后就可以你的爬虫之旅了