scrapy支持python2.7和python3.3以上版本,在windows下不支持python3版本(目前不支持)
官方建议安装scrapy在虚拟环境中(virtualenv)
Scrapy is written in pure Python and depends on a few key Python packages (among others):
lxml, an efficient XML and HTML parser
parsel, an HTML/XML data extraction library written on top of lxml,
w3lib, a multi-purpose helper for dealing with URLs and web page encodings
twisted, an asynchronous networking framework
cryptography and pyOpenSSL, to deal with various network-level security needs
安装scrapy前需要安装下列依赖:
sudo apt-get install python-dev python-pip libxml2-dev libxslt1-dev zlib1g-dev libffi-dev libssl-dev
如果想安装在python3版本中,还需要加上下列命令:
sudo apt-get install python3 python3-dev
python-dev, zlib1g-dev, libxml2-dev and libxslt1-dev are required for lxml
libssl-dev and libffi-dev are required for cryptography
Anaconda(一个科学计算的工具):
Using Anaconda is an alternative to using a virtualenv and installing with pip.
To install Scrapy using conda, run:
conda install -c conda-forge scrapy
installation guide 研读
最新推荐文章于 2019-08-01 23:57:47 发布