在UBUNTU上部署scrape过程

粗体为关键部分 

 

sudo apt-get install build-essential;
----安装此步会导致服务器很多服务升级，MYSQL,NAGIOS--谨慎--建议跳过这步，如果部署不成功再尝试部署这个。
 
sudo apt-get install python-dev;
sudo apt-get install libxml2-dev;
sudo apt-get install libxslt1-dev;
sudo apt-get install python-setuptools;

如安装源有问题，需更新安装源，/etc/apt/source.list

deb http://mirrors.aliyun.com/ubuntu/ raring main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ raring-security main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ raring-updates main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ raring-proposed main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ raring-backports main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ raring main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ raring-security main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ raring-updates main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ raring-proposed main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ raring-backports main restricted universe multiverse

安装Scrapy

sudo easy_install Scrapy

Finished processing dependencies for Scrapy

表示安装Scrapy成功

如安装中遭遇问题

pkg_resources.DistributionNotFound

解决方案如下

$ wget http://python-distribute.org/distribute_setup.py
$ python distribute_setup.py

 

测试

sudo scrapy startproject tutorial

如没有提示，且当前目录下出现tutorial 目录，代表成功。

 

如提示如下信息

UserWarning: You do not have the  service_identity module installed

则需要安装service_identity

sudo easy_install service_identity