Ubuntu安装 Scrapy框架

需要安装的包：python2.7、 Twisted、w3lib、lxml 或者libxml2、simplejson、pyopenssl、pycrypto、Scrapy。

python系统已经自带,Scrapy需要 python2.7。

1.安装 Twisted:

    sudo apt-get install python-twisted python-libxml2 python-simplejson

2.安装w3lib:

    sudo easy_install -U w3lib

3.安装pyOpenSSL:

  先安装openssl库：

  sudo apt-get install openssl

   sudo apt-get install libssl-dev build-essential zlibc zlib-bin libidn11-dev libidn11

  再安装pyOpenSSL:

 wget http://pypi.python.org/packages/source/p/pyOpenSSL/pyOpenSSL-0.13.tar.gz#md5=767bca18a71178ca353dff9e10941929

  tar -zxvf pyOpenSSL-0.13.tar.gz

  cd pyOpenSSL-0.13

  python setup.py build

  sudo python setup.py install

4.安装pycrypto:

  wget http://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.5.tar.gz#md5=783e45d4a1a309e03ab378b00f97b291

  tar -zxvf pycrypto-2.5.tar.gz

   cd pycrypto-2.5

  python setup.py build

  sudo python setup.py install

5.安装Scrapy:

  wget http://pypi.python.org/packages/source/S/Scrapy/Scrapy-0.14.3.tar.gz#md5=59f1225f7692f28fa0f78db3d34b3850

  tar -zxvf Scrapy-0.14.3.tar.gz

  cd Scrapy-0.14.3

   python setup.py build

  sudo python setup.py install

6.验证Scrapy安装：

  $scrapy