MAC python3.7安装pyspider遇到的一些坑

问题一

最后一句出现
File “c:\users\13733\appdata\local\programs\python\python37\lib\site-packages\pyspider\run.py”, line 231

async=True, get_object=False, no_input=False):

SyntaxError: invalid syntax

报错原因
python3.7中 async 是关键字，所以不能作为函数的参数名

解决办法
在pycharm中打开pyspider
我的pyspider路径是这样的：
/Users/yzchen/anaconda3/lib/python3.7/site-packages/pyspider/
更改这3个文件：pyspider>run.py，fetcher>tornado_fetcher.py
webui>app.py
ctrl+f查找async，把所有的async替换为shark

问题二

Error: Could not create web server listening on port 25555
[I 191026 18:37:14 result_worker:49] result_worker starting...
Error: Could not create web server listening on port 25555
[I 191026 18:37:15 processor:211] processor starting...
[I 191026 18:37:15 tornado_fetcher:638] fetcher starting...
[I 191026 18:37:15 scheduler:647] scheduler starting...
[I 191026 18:37:15 scheduler:782] scheduler.xmlrpc listening on 127.0.0.1:23333
[I 191026 18:37:15 scheduler:586] in 5m: new:0,success:0,retry:0,failed:0
[I 191026 18:37:15 app:84] webui exiting...
Traceback (most recent call last):
  File "/Users/yzchen/anaconda3/bin/pyspider", line 8, in <module>
    sys.exit(main())
  File "/Users/yzchen/anaconda3/lib/python3.7/site-packages/pyspider/run.py", line 754, in main
    cli()
  File "/Users/yzchen/anaconda3/lib/python3.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/Users/yzchen/anaconda3/lib/python3.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/Users/yzchen/anaconda3/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/yzchen/anaconda3/lib/python3.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/yzchen/anaconda3/lib/python3.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/Users/yzchen/anaconda3/lib/python3.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Users/yzchen/anaconda3/lib/python3.7/site-packages/pyspider/run.py", line 497, in all
    ctx.invoke(webui, **webui_config)
  File "/Users/yzchen/anaconda3/lib/python3.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/Users/yzchen/anaconda3/lib/python3.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Users/yzchen/anaconda3/lib/python3.7/site-packages/pyspider/run.py", line 384, in webui
    app.run(host=host, port=port)
  File "/Users/yzchen/anaconda3/lib/python3.7/site-packages/pyspider/webui/app.py", line 59, in run
    from .webdav import dav_app
  File "/Users/yzchen/anaconda3/lib/python3.7/site-packages/pyspider/webui/webdav.py", line 216, in <module>
    dav_app = WsgiDAVApp(config)
  File "/Users/yzchen/anaconda3/lib/python3.7/site-packages/wsgidav/wsgidav_app.py", line 134, in __init__
    _check_config(config)
  File "/Users/yzchen/anaconda3/lib/python3.7/site-packages/wsgidav/wsgidav_app.py", line 118, in _check_config
    raise ValueError("Invalid configuration:\n  - " + "\n  - ".join(errors))
ValueError: Invalid configuration:
  - Deprecated option 'domaincontroller': use 'http_authenticator.domain_controller' instead.

报错原因
wsgidav发布的3.x版本目前仍然是测试版，相对于2.x(例如2.4.1)更改了一些用法，上面报错的两个部分就是的。pyspider的3.0及以上版本在安装时，会默认安装wsgidav的3.x版（具体的版本可能会有偏差）

解决办法
其实上面错误信息已经提示该如何改了，不过那样改比较麻烦。可以换个方法，换回wsgidav的2.x版本就不会报错了。先把3.x版卸载，再装2.x版。下面是具体的卸载安装的命令

pip3 uninstall wsgidav
pip install wsgidav==2.4.1

问题三

Error: Could not create web server listening on port 25555
[I 191026 18:42:09 result_worker:49] result_worker starting...
[I 191026 18:42:10 processor:211] processor starting...
[I 191026 18:42:10 tornado_fetcher:638] fetcher starting...
Error: Could not create web server listening on port 25555
[I 191026 18:42:10 scheduler:647] scheduler starting...
[I 191026 18:42:10 scheduler:782] scheduler.xmlrpc listening on 127.0.0.1:23333
[I 191026 18:42:10 scheduler:586] in 5m: new:0,success:0,retry:0,failed:0
[I 191026 18:42:10 app:76] webui running on 0.0.0.0:5000
Error: Could not create web server listening on port 25555
Error: Could not create web server listening on port 25555
Error: Could not create web server listening on port 25555
Error: Could not create web server listening on port 25555
Error: Could not create web server listening on port 25555
Error: Could not create web server listening on port 25555
Error: Could not create web server listening on port 25555
Error: Could not create web server listening on port 25555
Error: Could not create web server listening on port 25555
Error: Could not create web server listening on port 25555
Error: Could not create web server listening on port 25555

错误原因
对应的25555端口被占用了

解决办法
找到该端口被谁占用，找到其pid，通过pid去kill掉即可

打开浏览器 0.0.0.0:5000
得到以下结果：

参考链接： https://www.jianshu.com/p/257b48a984ce
参考链接： https://blog.csdn.net/bakerp/article/details/93556164
参考链接： https://www.crifan.com/pyspider_run_error_could_not_create_web_server_listening_on_por_25555/
参考链接： https://blog.csdn.net/u011451186/article/details/88222328