安装scrapy
先安装一些依赖软件
yum install python-devel
yum install libffi-devel
yum install openssl-devel
然后安装pyopenssl库
pip install pyopenssl
注1:pip可能没有安装
首先安装epel扩展源:
sudo yum -y install epel-release
然后再安装pip
sudo yum -y install python-pip
注2:pip可能无法安装(网站被墙,需要修改下载源)
mkdir ~/.pip
vim ~/.pip/pip.conf
添加以下内容
[global]
index-url = http://mirrors.aliyun.com/pypi/simple/
[install]
trusted-host = mirrors.aliyun.com
安装xlml
yum install python-lxml
yum install libxml2-devel
yum install libxslt-devel
安装service-identity
pip install service-identity
安装twisted
pip install scrapy
安装scrapy
pip install scrapy -U
测试scrapy
scrapy bench