开发准备,安装scrapy框架
Scrapy是Python开发的一个快速、高层次的屏幕抓取和web抓取框架,用于抓取web站点并从页面中提取结构化的数据。Scrapy用途广泛,可以用于数据挖掘、监测和自动化测试。
** 1,基础环境搭建python开发 安装最新版python (省略)**
** 2,安装scrapy**
pip install scrapy
(或者)
pip3 install scrapy
安装界面
PS C:\WINDOWS\system32> pip install scrapy
Collecting scrapy
Using cached https://files.pythonhosted.org/packages/29/4b/585e8e111ffb01466c59281f34febb13ad1a95d7fb3919fd57c33fc732a5/Scrapy-1.7.3-py2.py3-none-any.whl
Collecting lxml; python_version != "3.4" (from scrapy)
Using cached https://files.pythonhosted.org/packages/bc/87/c3cecadcb5d7924cd71724b177343149cfc3609a89b197a991ac8593ed8c/lxml-4.4.1-cp37-cp37m-win_amd64.whl
Collecting w3lib>=1.17.0 (from scrapy)
Using cached https://files.pythonhosted.org/packages/6a/45/1ba17c50a0bb16bd950c9c2b92ec60d40c8ebda9f3371ae4230c437120b6/w3lib