OS X Python 3.7 openssl 终极冲突解决大法（PART 1）

OS X Python 3.7 openssl 终极冲突解决大法（PART 1）

---

2019/02/13 更新：
本文解决问题的思路及结论是对的：由于OS X 权限问题，Homebrew这个第三方工具未能完美的匹配OS X环境，但说实话，昨天写完这篇文章后，对本文中提出的解决办法，即不用Homebrew安装Python3，而改为从Python官网下载安装，我觉得是个不是办法的办法，毕竟没从根本上解决问题。
今天，继续研究探讨此问题，解决办法在PART 2中给出！

---

摘要：

  解决Mac OS X系统下，通过homebrew安装Python 3.7后，openssl所存在的各种问题，包括" import ssl “时出现的” ImportError: No module named _ssl “，用pip3安装beautifulsoup4时（ pip3 install beautifulsoup4 ）提示的” pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available. "等等。

问题的起源：

  1、假期过后，开始学习Ryan Mitchell所著《Python网络数据采集》，书中提到需要安装beautifulsoup4，利用pip3安装beautifulsoup4时，遇到如下问题：

$ pip3 install beautifulsoup4
pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Collecting beautifulsoup4
  Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/beautifulsoup4/

此处省略

pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.")) - skipping

  2、前期遗留问题尚未解决：OS X OPENSSL urlopen()函数 ERROR:root:code for hash md5 was not found.

   上述两个问题都指向一个关键问题：OS X下如何解决系统自带的python + openssl和homebrew安装的python + openssl的冲突。

• 系统环境
  Macbook Air + macOS Sierra 10.12.6
  系统自带python 2.7.10，安装目录：/System/Library/Framework/Python.framework/Versions/2.7
  Homebrew安装的python 3.7，安装目录：/usr/local/Cellar/python/3.7.2_1

搜索问题解决办法的过程

  通过在网上搜索，无外乎都指向两种答案：
    . 重新用homebrew安装openssl，并根据brew的提示执行后续操作；
    . 重新设置并编译安装python安装包，使得安装python3的时候自带openssl。

• 首先说第一种解决办法，即重新安装openssl：

$ brew install openssl
Updating Homebrew...
==> Auto-updated Homebrew!
Updated 2 taps (heroku/brew and homebrew/core).
==> New Formulae

此处省略...

Warning: openssl 1.0.2q is already installed, it's just not linked
You can use `brew link openssl` to link this version.

$ brew link openssl
Warning: Refusing to link macOS-provided software: openssl
If you need to have openssl first in your PATH run:
  echo 'export PATH="/usr/local/opt/openssl/bin:$PATH"' >> ~/.bash_profile

For compilers to find openssl you may need to set:
  export LDFLAGS="-L/usr/local/opt/openssl/lib"
  export CPPFLAGS="-I/usr/local/opt/openssl/include"

  因为我的openssl安装在"/usr/local/Cellar/openssl/1.0.2q/"目录中，所以：

$ echo 'export PATH="/usr/local/Cellar/openssl/1.0.2q/openssl/bin:$PATH"' >> ~/.bash_profile
$ export LDFLAGS="-L/usr/local/Cellar/openssl/1.0.2q/openssl/lib"
$ export CPPFLAGS="-I/usr/local/Cellar/openssl/1.0.2q/openssl/include"

  经过上面的操作，问题依旧，仍未解决pip3 install beautifulsoup4存在的openssl问题。
  继续搜索问题，下面的两个链接也都是将问题解决办法集中在重新安装openssl或者重新编译安装python上。
  “SSL module in Python is not available” when installing package with pip3

  pip3 installs inside virtual environment with python3.6 failing due to ssl module not available

  忽然有个想法，难道是系统自带openssl版本过低？
  继续搜索，搜索到Mac OS 10.12.2 重装升级 openssl，经测试，问题依旧，看来解决问题的方向不对啊。

• 第二个解决办法：重新编译安装python：
    一是本人水平有限，玩不转linux下的重新编译安装，二是直觉告诉我：linux也好OS X也罢，Python发展到今天，不应该动不动就需要重新编译安装吧？

  好吧，继续搜索解决办法。

  这时我做了个测试：

$ python3
Python 3.7.2 (default, Jan 13 2019, 12:51:54) 
[Clang 9.0.0 (clang-900.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ssl.py", line 98, in <module>
    import _ssl             # if we can't import it, let the error propagate
ImportError: dlopen(/usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload/_ssl.cpython-37m-darwin.so, 2): Library not loaded: /usr/local/opt/openssl/lib/libssl.1.0.0.dylib
  Referenced from: /usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload/_ssl.cpython-37m-darwin.so
  Reason: image not found
>>> exit()

$ python
Python 2.7.10 (default, Feb  7 2017, 00:08:15) 
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>> exit()

  这个测试证明了一个问题：系统自带的python2+openssl一切正常，问题还是在python3上。
  忽然冒出一个想法，同时也是受到mac上根治SSL的各种报错python3：的启发，既然我所遇到的一切问题都是基于一个出发点：Homebrew，那么，如果卸载Homebrew安装的Python，而改为从Python官网下载安装包呢？
  试试呗，首先卸载python

$ brew uninstall python --force

  然后去官网下载python3.7安装包，官网地址：Python 3.7.2

  接下来就是安装，安装过程一切顺利，不再赘述。

  然后是重新安装pip3

$ python3 get-pip.py

  然后是回到原点，安装beautifulsoup4。

$ pip3 install beautifulsoup4
Collecting beautifulsoup4
  Downloading https://files.pythonhosted.org/packages/1d/5d/3260694a59df0ec52f8b4883f5d23b130bc237602a1411fa670eae12351e/beautifulsoup4-4.7.1-py3-none-any.whl (94kB)
    100% |████████████████████████████████| 102kB 156kB/s 
Collecting soupsieve>=1.2 (from beautifulsoup4)
  Downloading https://files.pythonhosted.org/packages/bf/b3/2473abf05c4950c6a829ed5dcbc40d8b56d4351d15d6939c8ffb7c6b1a14/soupsieve-1.7.3-py2.py3-none-any.whl
Installing collected packages: soupsieve, beautifulsoup4
Successfully installed beautifulsoup4-4.7.1 soupsieve-1.7.3

  哈哈，居然一切顺利，成功了。
  再测试python3中的ssl。

$ python3
Python 3.7.2 (v3.7.2:9a3ffc0492, Dec 24 2018, 02:44:43) 
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>> 

一切顺利！

结论：

我认为，应该是OS X的权限问题，导致系统自带的python + openssl和Homebrew安装的Python3.7有冲突，同时，也是Homebrew这个第三方工具未能完美的匹配OS X环境，才导致上述问题的发生，所以：
建议OS X环境下安装Python 3，还是从官网下载安装包，避免使用Homebrew安装。