python安装scrapy失败_安装scrapy，以及出现的错误解决。

首先我是在python3的环境上面完成的。我保留了python2的版本，然后安装python3的版本。然后在安装scrapy的过程中出现的错误，以及切换python版本后出现的错误。

一、安装python3

cd /usr/local/src/

wget https://www.python.org/ftp/python/3.4.2/Python-3.4.2.tgz

tar -xf Python-3.4.2.tgz

cd Python-3.4.2

./configure --prefix=/usr/local/python3

make && make install

##将python做一个备份，然后把python3的建立一个软连接

mv /usr/bin/python /usr/bin/pythonbak

ln -fs /usr/local/python3/bin/python3 /usr/bin/python

再在终端进入python交互模式，出现的是python3了。

[root@lsf ~]# python

Python 3.4.2 (default, Mar 13 2018, 11:37:48)

[GCC 4.8.5 20150623 (Red Hat 4.8.5-16)] on linux

Type "help", "copyright", "credits" or "license" for more information.

##python升级到3后，yum无法正常使用，此时需要改一个文件

[root@lsf ~]# cat /usr/bin/yum

#!/usr/bin/python2.7

将第一行改为python2.7

二、安装scrapy

##建立pip3的软连接

ln -fs /usr/local/python3/bin/pip3 /usr/bin/pip3

##yum安装依赖包,可以解决编译过程中出现的许多错误

yum install -y gcc openssl-devel libxml2-devel libxslt-devel bzip2-devel bzip2* libffi-devel python-devel openssl-devel pycrypto

##用pip3安装

pip3 install twisted --upgrade

pip3 install cryptography pycrypto

##安装scrapy

pip3 install scrapy

##建立scrapy软连接

ln -fs /usr/local/python3/bin/scrapy /usr/bin/scrapy

三、测试scrapy命令

[root@lsf ~]# scrapy

Scrapy 1.5.0 - no active project

Usage:

scrapy [options] [args]

Available commands:

bench Run quick benchmark test

fetch Fetch a URL using the Scrapy downloader

genspider Generate new spider using pre-defined templates

runspider Run a self-contained spider (without creating a project)

settings Get settings values

shell Interactive scraping console

startproject Create new project

version Print Scrapy version

view Open URL in browser, as seen by Scrapy

[ more ] More commands available when run from project directory

Use "scrapy -h" to see more info about a command

##创建一个scrapy项目

scrapy startproject myfirstpjt

##可能会报错ImportError: cannot import name 'certificate_transparency'

pip3 install pip3 --upgrade

##再次安装scrapy

pip3 install scrapy

##创建项目成功后，会出现下面的文件

[root@lsf test_scrapy]# cd myfirstpjt/

[root@lsf myfirstpjt]# ls

myfirstpjt scrapy.cfg

[root@lsf myfirstpjt]# cd myfirstpjt/

[root@lsf myfirstpjt]# ls

__init__.py items.py middlewares.py pipelines.py __pycache__ settings.py spiders

然后就可以直接使用scrapy命令进行项目管理了。