大数据爬虫基础（三）Scrapy在ubuntu 16.04下的安装

Scrapy ubuntu下安装

系统：ubuntu 16.04 no gui

依赖包及依赖包的依赖包：
下列的安装步骤假定您已经安装好下列程序:
http://scrapy.org/
Python 2.7
Python Package: pip and setuptools. 现在 pip 依赖 setuptools ，如果未安装，则会自动安装 setuptools 。
lxml. 大多数Linux发行版自带了lxml。如果缺失，请查看http://lxml.de/installation.html
OpenSSL. 除了Windows(请查看 平台安装指南)之外的系统都已经提供。
您可以使用pip来安装Scrapy(推荐使用pip来安装Python package).


使用pip安装:
小写scrapy不是大写，官网是小写
http://scrapy-chs.readthedocs.org/zh_CN/latest/intro/install.html
pip install scrapy


1、pip,easy_install
Ubuntu下安装pip的方法
 http://www.2cto.com/os/201305/213725.html
安装pip的方法：
Install pip and virtualenv for Ubuntu 10.10 Maverick and newer
 
$ sudo apt-get install python-pip python-dev build-essential 
$ sudo pip install --upgrade pip 
$ sudo pip install --upgrade virtualenv 
For older versions of Ubuntu
 
Install Easy Install
$ sudo apt-get install python-setuptools python-dev build-essential 
Install pip
$ sudo easy_install pip 
Install virtualenv
$ sudo pip install --upgrade virtualenv 
sudo apt-get install python-setuptools python-dev build-essential




2、lxml
先安装依赖包，否则装不上，报错：x86_64-linux-gnu-gcc error
装：apt-get install -y libxml2-dev libxslt1-dev zlib1g-dev python3-pip


或者装：apt-get install build-essential autoconf libtool pkg-config python-opengl python-imaging python-pyrex python-pyside.qtopengl idle-python2.7 qt4-dev-tools qt4-designer libqtgui4 libqtcore4 libqt4-xml libqt4-test libqt4-script libqt4-network libqt4-dbus python-qt4 python-qt4-gl libgle3 python-dev


success后
再pip install lxml
success


3、cryptography及其依赖包
直接pip install scrapy会报 cryptography和cffi的错误，安装以下依赖包：
https://cryptography.io/en/latest/installation/#building-cryptography-on-linux
apt-get install build-essential libssl-dev libffi-dev python-dev
success


4、Scrapy
pip install scrapy
success




5、可选包：
pip install pymongo
pip install pillow
pip install pycrypto


6、报错参考
http://stackoverflow.com/questions/22073516/failed-to-install-python-cryptography-package-with-pip-and-setup-Python
http://stackoverflow.com/questions/27130286/error-command-x86-64-linux-gnu-gcc-failed-with-exit-status-1-in-virtualenv
http://www.cnblogs.com/lyroge/archive/2013/02/22/2922515.html