背景
项目中需要做个网络爬虫,想使用Python爬虫框架pyspider,遇到安装报错如下
# 执行安装
>pip install pyspider
ERROR: Command errored out with exit status 10:
command: 'd:\java\pyworkspace\spider\venv\scripts\python.exe' -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\milla\\AppData\\Local\\Temp\\pip-install-4zai8mou\
\pycurl_b94259d1e5594c498458083d4b02bc99\\setup.py'"'"'; __file__='"'"'C:\\Users\\milla\\AppData\\Local\\Temp\\pip-install-4zai8mou\\pycurl_b94259d1e5594c498458083d4b02bc99\\setup.py'"'
"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base '
C:\Users\milla\AppData\Local\Temp\pip-pip-egg-info-adu6pbve'
cwd: C:\Users\milla\AppData\Local\Temp\pip-install-4zai8mou\pycurl_b94259d1e5594c498458083d4b02bc99\
Complete output (1 lines):
Please specify --curl-dir=/path/to/built/libcurl
----------------------------------------
ERROR: Command errored out with exit status 10: python setup.py egg_info Check the logs for full command output.
字面提示是在安装的过程中需要指定一个依赖项(),这个确实没有Maven管理来的方便...
于是从PythonLibs官网中获取依赖并自行下载到本地
PS:因为依赖项太多,需要依据提示中少了的libcur去查找【ctl+F,在弹出的框中搜索libcurl】,然后找到和自己适配的版本 ,笔者是python38 32位的,所以选择如下
如果版本不兼容的话会报错
ERROR: pycurl-7.43.0.4-cp38-cp38-win_amd64.whl is not a supported wheel on this platform.
再次执行>pip install pyspider,安装成功