【Scrapy爬虫学习笔记一】安装scrapy

安装scrapy

先安装一些依赖软件

yum install python-devel
yum install libffi-devel
yum install openssl-devel

然后安装pyopenssl库

pip install pyopenssl

注1：pip可能没有安装

首先安装epel扩展源：

sudo yum -y install epel-release

然后再安装pip

sudo yum -y install python-pip

注2：pip可能无法安装（网站被墙，需要修改下载源）

mkdir ~/.pip 
vim ~/.pip/pip.conf
添加以下内容

[global]
index-url = http://mirrors.aliyun.com/pypi/simple/
[install]
trusted-host = mirrors.aliyun.com

安装xlml

yum install python-lxml
yum install libxml2-devel
yum install libxslt-devel

安装service-identity

pip install service-identity

安装twisted

pip install scrapy

安装scrapy

pip install scrapy -U

测试scrapy

scrapy bench