mac安装pyspider的步骤和问题

一般来说，直接使用pip install pyspider会失败，信息为

 ERROR: Complete output from command python setup.py egg_info:
    ERROR: Using curl-config (libcurl 7.54.0)
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/tmp/pip-install-wrZqoT/pycurl/setup.py", line 913, in <module>
        ext = get_extension(sys.argv, split_extension_source=split_extension_source)
      File "/private/tmp/pip-install-wrZqoT/pycurl/setup.py", line 582, in get_extension
        ext_config = ExtensionConfiguration(argv)
      File "/private/tmp/pip-install-wrZqoT/pycurl/setup.py", line 99, in __init__
        self.configure()
      File "/private/tmp/pip-install-wrZqoT/pycurl/setup.py", line 316, in configure_unix
        specify the SSL backend manually.''')
    __main__.ConfigurationError: Curl is configured to use SSL, but we have not been able to determine which SSL backend it is using. Please see PycURL documentation for how to specify the SSL backend manually.
    ----------------------------------------
ERROR: Command "python setup.py egg_info" failed with error code 1 in /private/tmp/pip-install-wrZqoT/pycurl/

如果安装过openssl则略过，未安装过则使用brew install openssl安装openssl 

接下来使用语句find /usr/local -name ssl.h来找到ssl.h

然后使用下面三个语句添加环境变量

export PYCURL_SSL_LIBRARY=openssl
export LDFLAGS=-L/usr/local/Cellar/openssl/1.0.2r/lib
export CPPFLAGS=-I/usr/local/Cellar/openssl/1.0.2r/include

配置完变量之后，使用语句来安装pycurl

pip install pycurl  

 接下来就是最后一步pip install pyspider

安装成功！

 

安装成功后，并不是一帆风顺，启动又来了问题，报错信息如下：

raise ValueError("Invalid configuration:\n  - " + "\n  - ".join(errors))
ValueError: Invalid configuration:
  - Deprecated option 'domaincontroller': use 'http_authenticator.domain_controller' instead.

查了好久百度，原来是因为python版本为2.7，而安装pyspider默认安装了wsgidav的3.0版本，如下

而python2.7和wsgidav的3.0版本不兼容，所以导致上述的报错，使用下面语句重新安装即可：

pip uninstall wsgidav
pip install wsgidav==2.4.1

安装之后，有可能报错：

Error: Could not create web server listening on port 25555

这是因为在之前已经启动了phantomjs，使用命令lsof -i:25555找到PID然后kill掉即可。

最后运行成功：