distribute_crawler项目实战

安装scrapy

https://scrapy-chs.readthedocs.org/zh_CN/latest/intro/install.html

使用pip安装:

pip install Scrapy

Ubuntu 软件包

• 把Scrapy签名的GPG密钥添加到APT的钥匙环中:

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 627220E7

• 执行如下命令，创建 /etc/apt/sources.list.d/scrapy.list 文件:

echo 'deb http://archive.scrapy.org/ubuntu scrapy main' | sudo tee /etc/apt/sources.list.d/scrapy.list

• 更新包列表并安装 scrapy-0.25:

sudo apt-get update && sudo apt-get install scrapy-0.25

安装redispy

sudo easy_install redis  

安装pymongo

sudo pip install pymongo==2.7.2

安装graphite

http://blog.csdn.net/u011008734/article/details/47166469
注意之后 需要每次启动 carb