redhat6.5部署python爬虫项目（一）

因为最近工作开始接触python爬虫，所以将学习历程记录下来。一是为了方便以后查阅，二是为了帮助遇到相同困难的各位。

VM安装redhat6.5的过程我就跳过了，网上全是教程。

一、安装python3.7

首先需要配置yum源

目前，官方把yum源删除了，国内的163等镜像源也没有6.0的内容。所以使用yum命令安装软件包基本都是失败，因此解决办法就是更换yum源。

我先说yum源的事情是因为redhat6.5自带的时python2.6。而现在的开发基本都是python3.X。而在安装python3环境时，需要yum去装一些依赖环境，才能正确安装python3.X。

# rpm -qa |grep yum
// 查看yum

# rpm -qa |grep yum |xargs rpm -e --nodeps
//删除yum

# rpm -qa |grep yum
//再次查看yum的内容，如果为空则表明删除成功

yum 所需要安装的rpm文件我已经放到百度网盘了，需要可以自取。

链接：https://pan.baidu.com/s/189cF1PmMztGhR5Ygyq0HZg         提取码：h2nm 

里面共四个文件：

下载后转移到redhat上，不清楚的可以百度。我用的时winSCP给服务器传输文件。

安装rpm文件：

注意：先去安装 python-iniparse-0.3.1-2.1.el6.noarch.rpm ，如果显示已有就忽略。

rpm -ivh python-iniparse-0.3.1-2.1.el6.noarch.rpm

接下来把其余三个文件一起安装，因为会有互相依赖的问题，所以一定要注意同时安装。

rpm -ivh yum-3.2.29-40.el6.centos.noarch.rpm yum-metadata-parser-1.1.2-16.el6.x86_64.rpm yum-plugin-fastestmirror-1.1.30-14.el6.noarch.rpm

安装成功后开始更换yum源，下载以下两个repo文件。

wget -O /etc/yum.repos.d/CentOS-Base.repo http://file.kangle.odata.cc/repo/Centos-6.repo

wget -O /etc/yum.repos.d/epel.repo http://file.kangle.odata.cc/repo/epel-6.repo

然后执行

yum clean all
yum makecache

查看yum版本看是否安装成功

yum -v

如果成功接下来开始安装python3.X版本。

安装Python3.7

安装依赖包

yum -y install zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel readline-devel tk-devel gdbm-devel db4-devel libpcap-devel xz-devel
yum install libffi-devel -y
yum -y install zlib*

尽量安装足够多的依赖包，不然报错又开始受罪。切记切记！！！

下载解压：

wget https://www.python.org/ftp/python/3.7.2/Python-3.7.2.tgz
tar -xzvf Python-3.7.2.tgz
cd Python-3.7.2

编译安装：

./configure --prefix=/usr/local/python3
make
make insatll

此时千万不要着急这样做，因为会出现pip3版本过低的情况导致以下报错：

pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Collecting virtualenv
  Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/virtualenv/
  Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/virtualenv/
  Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/virtualenv/
  Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/virtualenv/
  Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/virtualenv/
  Could not fetch URL https://pypi.org/simple/virtualenv/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/virtualenv/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.")) - skipping
  Could not find a version that satisfies the requirement virtualenv (from versions: )
No matching distribution found for virtualenv
pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.")) - skipping

因为此时的openssl版本只是1.0.1，而python3.7需要的openssl版本为1.0.2或者1.1.x，因此需要对openssl进行升级，并重新编译python3.7.2

1、下载openssl并编译安装：

wget https://www.openssl.org/source/openssl-1.1.1a.tar.gz
tar -zxvf openssl-1.1.1a.tar.gz
cd openssl-1.1.1a
./config --prefix=/usr/local/openssl no-zlib #不需要zlib
make
make install

2、备份：

mv /usr/bin/openssl /usr/bin/openssl.bak
mv /usr/include/openssl/ /usr/include/openssl.bak

3、新版配置：

//将安装好的openssl 的openssl命令软连到/usr/bin/openssl
# ln -s /usr/local/openssl/include/openssl /usr/include/openssl

//软链到升级后的libssl.so
# ln -s /usr/local/openssl/lib/libssl.so.1.1 /usr/local/lib64/libssl.so

//将安装好的openssl命令软连到/usr/bin/openssl
ln -s /usr/local/openssl/bin/openssl /usr/bin/openssl

4、修改系统配置：

//写入openssl库文件的搜索路径
echo "/usr/local/openssl/lib" >> /etc/ld.so.conf

//使修改后的/etc/ld.so.conf生效 
ldconfig -v

5、查看openssl版本：出现一下即为成功

openssl version
OpenSSL 1.1.1-pre8 (beta) 20 Jun 2018

重新编译安装python

注意：需要指定openssl的安装路径，--with-openssl

./configure --prefix=/usr/local/python3 --with-openssl=/usr/local/openssl
make
make install

最后测试pip3安装是否成功

pip3 -v


pip3 install pymysql

安装成功，大功告成。

Collecting pymysql
  Downloading https://files.pythonhosted.org/packages/4f/52/a115fe175028b058df353c5a3d5290b71514a83f67078a6482cff24d6137/PyMySQL-1.0.2-py3-none-any.whl (43kB)
    100% |████████████████████████████████| 51kB 879kB/s 
Installing collected packages: pymysql
Successfully installed pymysql-1.0.2

参考：

https://blog.csdn.net/qq_40640228/article/details/112066789

https://blog.51cto.com/u_13544424/2149473