reducto安装requirement.txt 踩过的坑

安装requirement.txt 踩过的坑

这几天在尝试reducto (https://github.com/coco-create/reducto )项目代码实施的过程中安装requirements 的过程中遇到了很多坑爹的问题，特此记录。

问题一：虚拟环境

一开始没有使用虚拟环境，直接在base中安装库，造成包和环境混乱。

问题二：安装gpustat

Collecting gpustat==0.6.0
  Downloading http://mirrors.aliyun.com/pypi/packages/b4/69/d8c849715171aeabd61af7da080fdc60948b5a396d2422f1f4672e43d008/gpustat-0.6.0.tar.gz (78 kB)
     |████████████████████████████████| 78 kB 4.5 MB/s 
    ERROR: Command errored out with exit status 1:
     command: /home/coco/anaconda3/envs/conda_env820/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-m8a70_wt/gpustat_30f3cd6576c64f5b811d71718fd21446/setup.py'"'"'; __file__='"'"'/tmp/pip-install-m8a70_wt/gpustat_30f3cd6576c64f5b811d71718fd21446/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-a94szqfx
         cwd: /tmp/pip-install-m8a70_wt/gpustat_30f3cd6576c64f5b811d71718fd21446/
    Complete output (33 lines):
    WARNING: The repository located at mirrors.aliyun.com is not a trusted or secure host and is being ignored. If this repository is available via HTTPS we recommend you use HTTPS instead, otherwise you may silence this warning and allow it anyway with '--trusted-host mirrors.aliyun.com'.
    ERROR: Could not find a version that satisfies the requirement pytest-runner
    ERROR: No matching distribution found for pytest-runner
    Traceback (most recent call last):
      File "/home/coco/anaconda3/envs/conda_env820/lib/python3.6/site-packages/setuptools/installer.py", line 75, in fetch_build_egg
        subprocess.check_call(cmd)
      File "/home/coco/anaconda3/envs/conda_env820/lib/python3.6/subprocess.py", line 311, in check_call
        raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command '['/home/coco/anaconda3/envs/conda_env820/bin/python', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpl4cewqkr', '--quiet', 'pytest-runner']' returned non-zero exit status 1.
    
    The above exception was the direct cause of the following exception:
    
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-m8a70_wt/gpustat_30f3cd6576c64f5b811d71718fd21446/setup.py", line 123, in <module>
        zip_safe=False,
      File "/home/coco/anaconda3/envs/conda_env820/lib/python3.6/site-packages/setuptools/__init__.py", line 152, in setup
        _install_setup_requires(attrs)
      File "/home/coco/anaconda3/envs/conda_env820/lib/python3.6/site-packages/setuptools/__init__.py", line 147, in _install_setup_requires
        dist.fetch_build_eggs(dist.setup_requires)
      File "/home/coco/anaconda3/envs/conda_env820/lib/python3.6/site-packages/setuptools/dist.py", line 689, in fetch_build_eggs
        replace_conflicting=True,
      File "/home/coco/anaconda3/envs/conda_env820/lib/python3.6/site-packages/pkg_resources/__init__.py", line 768, in resolve
        replace_conflicting=replace_conflicting
      File "/home/coco/anaconda3/envs/conda_env820/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1051, in best_match
        return self.obtain(req, installer)
      File "/home/coco/anaconda3/envs/conda_env820/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1063, in obtain
        return installer(requirement)
      File "/home/coco/anaconda3/envs/conda_env820/lib/python3.6/site-packages/setuptools/dist.py", line 745, in fetch_build_egg
        return fetch_build_egg(self, req)
      File "/home/coco/anaconda3/envs/conda_env820/lib/python3.6/site-packages/setuptools/installer.py", line 77, in fetch_build_egg
        raise DistutilsError(str(e)) from e
    distutils.errors.DistutilsError: Command '['/home/coco/anaconda3/envs/conda_env820/bin/python', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpl4cewqkr', '--quiet', 'pytest-runner']' returned non-zero exit status 1.
    ----------------------------------------
WARNING: Discarding http://mirrors.aliyun.com/pypi/packages/b4/69/d8c849715171aeabd61af7da080fdc60948b5a396d2422f1f4672e43d008/gpustat-0.6.0.tar.gz#sha256=f69135080b2668b662822633312c2180002c10111597af9631bb02e042755b6c (from http://mirrors.aliyun.com/pypi/simple/gpustat/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement gpustat==0.6.0
ERROR: No matching distribution found for gpustat==0.6.0

• 问题根源：

	ERROR: Could not find a version that satisfies the requirement pytest-runner
    ERROR: No matching distribution found for pytest-runner

-解决方法：

	pip3 install pytest-runner

问题三：安装importlib

Collecting importlib==1.0.4
  Downloading http://mirrors.aliyun.com/pypi/packages/31/77/3781f65cafe55480b56914def99022a5d2965a4bb269655c89ef2f1de3cd/importlib-1.0.4.zip (7.1 kB)
    ERROR: Command errored out with exit status 1:
     command: /home/coco/anaconda3/envs/conda_env820/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-l_3zn6l9/importlib_4aa51a3ece2348f4a3a25fddc6f740ae/setup.py'"'"'; __file__='"'"'/tmp/pip-install-l_3zn6l9/importlib_4aa51a3ece2348f4a3a25fddc6f740ae/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-rov4bnhu
         cwd: /tmp/pip-install-l_3zn6l9/importlib_4aa51a3ece2348f4a3a25fddc6f740ae/
    Complete output (11 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/home/coco/anaconda3/envs/conda_env820/lib/python3.6/site-packages/setuptools/__init__.py", line 10, in <module>
        import distutils.core
      File "/home/coco/anaconda3/envs/conda_env820/lib/python3.6/distutils/core.py", line 16, in <module>
        from distutils.dist import Distribution
      File "/home/coco/anaconda3/envs/conda_env820/lib/python3.6/distutils/dist.py", line 19, in <module>
        from distutils.util import check_environ, strtobool, rfc822_escape
      File "/home/coco/anaconda3/envs/conda_env820/lib/python3.6/distutils/util.py", line 9, in <module>
        import importlib.util
    ModuleNotFoundError: No module named 'importlib.util'
    ----------------------------------------
WARNING: Discarding http://mirrors.aliyun.com/pypi/packages/31/77/3781f65cafe55480b56914def99022a5d2965a4bb269655c89ef2f1de3cd/importlib-1.0.4.zip#sha256=b6ee7066fea66e35f8d0acee24d98006de1a0a8a94a8ce6efe73a9a23c8d9826 (from http://mirrors.aliyun.com/pypi/simple/importlib/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement importlib==1.0.4
ERROR: No matching distribution found for importlib==1.0.4

折腾很久，各种百度+google +CSDN + 知乎，尝试过换源、升级pip、使用whl文件/源码安装等等方式均失败，很想骂人xxxxxx。

• 问题根源：手动使用源码安装时发现 readme中如下：（python和包的版本不匹配）

Purpose
========
This package contains the code from importlib as found in Python 2.7. It is
provided so that people who wish to use importlib.import_module() with a
version of Python prior to 2.7 or in 3.0 have the function readily available.
The code in no way deviates from what can be found in the Python 2.7 standard
library.
For documentation, see the importlib docs_ for Python 2.7.
… _importlib docs: http://docs.python.org/2.7/library/importlib.html
Notice about maintenance
========================

• 问题解决：重新建一个python版本为2.7的虚拟环境

问题四：安装pybgs

Building wheels for collected packages: pybgs
  Building wheel for pybgs (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-57jmc1zb/pybgs_eb601cc1b6764da9b1ccc357c001ef44/setup.py'"'"'; __file__='"'"'/tmp/pip-install-57jmc1zb/pybgs_eb601cc1b6764da9b1ccc357c001ef44/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-x6roxd2z
       cwd: /tmp/pip-install-57jmc1zb/pybgs_eb601cc1b6764da9b1ccc357c001ef44/
  Complete output (171 lines):
  running bdist_wheel
  running build
  running build_ext
  Preparing the build environment
  Configuring cmake project
  cmake -H/tmp/pip-install-57jmc1zb/pybgs_eb601cc1b6764da9b1ccc357c001ef44 -Bbuild/temp.linux-x86_64-3.6 -DPYTHON_EXECUTABLE=/usr/bin/python3 -DBGS_CORE_STATIC=ON -DBGS_PYTHON_SUPPORT=ON -DBGS_PYTHON_ONLY=ON -DBGS_PYTHON_VERSION=3.6
  -- The C compiler identification is GNU 7.5.0
  -- The CXX compiler identification is GNU 7.5.0
  -- Check for working C compiler: /usr/bin/cc
  -- Check for working C compiler: /usr/bin/cc -- works
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- Check for working CXX compiler: /usr/bin/c++
  -- Check for working CXX compiler: /usr/bin/c++ -- works
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  --
  -- BGS_PYTHON_SUPPORT: ON
  -- BGS_PYTHON_ONLY:    ON
  -- BGS_CORE_STATIC:    ON
  -- PYTHON VERSION: 3.6
  -- Found OpenCV: /usr/local (found version "3.4.11")
  --
  -- OpenCV library status:
  --     version: 3.4.11
  --     libraries: opencv_calib3d;opencv_core;opencv_dnn;opencv_features2d;opencv_flann;opencv_highgui;opencv_imgcodecs;opencv_imgproc;opencv_ml;opencv_objdetect;opencv_photo;opencv_shape;opencv_stitching;opencv_superres;opencv_video;opencv_videoio;opencv_videostab
  --     include path: /usr/local/include;/usr/local/include/opencv
  
  -- Found PythonInterp: /usr/bin/python3 (found suitable version "3.6.9", minimum required is "3.6")
  -- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython3.6m.so
  -- pybind11 v2.3.dev1
  --
  -- Python library status:
  --     executable: /usr/bin/python3
  --     library: /usr/lib/x86_64-linux-gnu/libpython3.6m.so
  --     include path: /usr/include/python3.6m
  -- NUMPY_INCLUDE_DIR: /home