python实验实践【1】python包的管理

一般情况下，github上repo会在README文件中给出实验代码跑通依赖的包库。这通常是一个繁琐且复杂的工作，往往需要很长时间。本篇博客介绍跑通别人代码之前所需要进行的包库安装工作具体细节。

包的信息

实验所需要的安装包所含的信息涉及：包的名称与包的版本。很多情况包的版本不对会导致代码运行失败。这种情况基本无法通过改代码来实现，往往采用更改安装包的版本进行调试。

特定python安装包的功能实现有时需要依赖特定的其他python安装包，这种功能实现的依赖关系被称为：包的依赖关系。

无论是单一包的名称与版本，还是多个包之间的依赖关系，都可以使用单一bash语句查看。

包的版本信息

通过调用下面语句可以查看ray包的版本信息

pip show ray

返回结果如下：

Name: ray
Version: 0.8.7
Summary: A system for parallel and distributed Python that unifies the ML ecosystem.
Home-page: https://github.com/ray-project/ray
Author: Ray Team
Author-email: ray-dev@googlegroups.com
License: Apache 2.0
Location: /home/liujw/miniconda3/envs/neurocard/lib/python3.7/site-packages
Requires: filelock, colorama, requests, opencensus, protobuf, colorful, google, aioredis, msgpack, numpy, click, aiohttp, py-spy, pyyaml, jsonschema, grpcio, redis, prometheus-client, gpustat
Required-by: 

返回结果种可以看到ray这个包的创作主页，作者，作者邮箱，安装的位置和依赖的包库信息等。

包库依赖关系

调用pip show ray可以查看ray包依赖包的名称，但缺少版本信息。若想查看更细致的包库依赖信息可以调用下面语句。

pipdeptree -p ray

返回结果如下：

ray==0.8.7
  - aiohttp [required: Any, installed: 3.7.4.post0]
    - async-timeout [required: >=3.0,<4.0, installed: 3.0.1]
    - attrs [required: >=17.3.0, installed: 21.2.0]
    - chardet [required: >=2.0,<5.0, installed: 4.0.0]
    - multidict [required: >=4.5,<7.0, installed: 5.2.0]
    - typing-extensions [required: >=3.6.5, installed: 3.10.0.2]
    - yarl [required: >=1.0,<2.0, installed: 1.7.0]
      - idna [required: >=2.0, installed: 3.2]
      - multidict [required: >=4.0, installed: 5.2.0]
      - typing-extensions [required: >=3.7.4, installed: 3.10.0.2]
  - aioredis [required: Any, installed: 1.3.1]
    - async-timeout [required: Any, installed: 3.0.1]
    - hiredis [required: Any, installed: 2.0.0]
  - click [required: >=7.0, installed: 8.0.1]
    - importlib-metadata [required: Any, installed: 4.8.1]
      - typing-extensions [required: >=3.6.4, installed: 3.10.0.2]
      - zipp [required: >=0.5, installed: 3.6.0]
  - colorama [required: Any, installed: 0.4.4]
  - colorful [required: Any, installed: 0.5.4]
  - filelock [required: Any, installed: 3.0.12]
  - google [required: Any, installed: 3.0.0]
    - beautifulsoup4 [required: Any, installed: 4.10.0]
      - soupsieve [required: >1.2, installed: 2.2.1]
  - gpustat [required: Any, installed: 0.4.1]
    - blessings [required: >=1.6, installed: 1.7]
      - six [required: Any, installed: 1.16.0]
    - nvidia-ml-py3 [required: >=7.352.0, installed: 7.352.0]
    - psutil [required: Any, installed: 5.0.0]
    - six [required: Any, installed: 1.16.0]
  - grpcio [required: >=1.28.1, installed: 1.40.0]
    - six [required: >=1.5.2, installed: 1.16.0]
  - jsonschema [required: Any, installed: 3.2.0]
    - attrs [required: >=17.4.0, installed: 21.2.0]
    - importlib-metadata [required: Any, installed: 4.8.1]
      - typing-extensions [required: >=3.6.4, installed: 3.10.0.2]
      - zipp [required: >=0.5, installed: 3.6.0]
    - pyrsistent [required: >=0.14.0, installed: 0.18.0]
    - setuptools [required: Any, installed: 58.2.0]
    - six [required: >=1.11.0, installed: 1.16.0]
  - msgpack [required: >=1.0.0,<2.0.0, installed: 1.0.2]
  - numpy [required: >=1.16, installed: 1.18.4]
  - opencensus [required: Any, installed: 0.7.13]
    - google-api-core [required: >=1.0.0,<2.0.0, installed: 1.31.3]
      - google-auth [required: >=1.25.0,<2.0dev, installed: 1.35.0]
        - cachetools [required: >=2.0.0,<5.0, installed: 4.2.4]
        - pyasn1-modules [required: >=0.2.1, installed: 0.2.8]
          - pyasn1 [required: >=0.4.6,<0.5.0, installed: 0.4.8]
        - rsa [required: >=3.1.4,<5, installed: 4.7.2]
          - pyasn1 [required: >=0.1.3, installed: 0.4.8]
        - setuptools [required: >=40.3.0, installed: 58.2.0]
        - six [required: >=1.9.0, installed: 1.16.0]
      - googleapis-common-protos [required: >=1.6.0,<2.0dev, installed: 1.53.0]
        - protobuf [required: >=3.12.0, installed: 3.17.3]
          - six [required: >=1.9, installed: 1.16.0]
      - packaging [required: >=14.3, installed: 21.0]
        - pyparsing [required: >=2.0.2, installed: 2.4.7]
      - protobuf [required: >=3.12.0,<3.18.0, installed: 3.17.3]
        - six [required: >=1.9, installed: 1.16.0]
      - pytz [required: Any, installed: 2021.3]
      - requests [required: >=2.18.0,<3.0.0dev, installed: 2.26.0]
        - certifi [required: >=2017.4.17, installed: 2021.10.8]
        - charset-normalizer [required: ~=2.0.0, installed: 2.0.6]
        - idna [required: >=2.5,<4, installed: 3.2]
        - urllib3 [required: >=1.21.1,<1.27, installed: 1.26.7]
      - setuptools [required: >=40.3.0, installed: 58.2.0]
      - six [required: >=1.13.0, installed: 1.16.0]
    - opencensus-context [required: ==0.1.2, installed: 0.1.2]
  - prometheus-client [required: >=0.7.1, installed: 0.11.0]
  - protobuf [required: >=3.8.0, installed: 3.17.3]
    - six [required: >=1.9, installed: 1.16.0]
  - py-spy [required: >=0.2.0, installed: 0.3.9]
  - pyyaml [required: Any, installed: 3.10]
  - redis [required: >=3.3.2,<3.5.0, installed: 3.4.1]
  - requests [required: Any, installed: 2.26.0]
    - certifi [required: >=2017.4.17, installed: 2021.10.8]
    - charset-normalizer [required: ~=2.0.0, installed: 2.0.6]
    - idna [required: >=2.5,<4, installed: 3.2]
    - urllib3 [required: >=1.21.1,<1.27, installed: 1.26.7]

返回结果中展示了ray都依赖哪些python包库，以及它们对应的版本。当目前安装的python包库发生冲突时会有报错。

Warning!!! Possibly conflicting dependencies found:
* wandb==0.8.36
 - gql [required: ==0.2.0, installed: 0.1.0]

例如这里gql需要的版本是0.2.0但是目前安装的版本是0.1.0。

包的实践

完成项目后，将项目开源的第一步是将包库信息汇总成文档。pip提供了一个语句可以实现提取当前环境python包库及版本信息，以及按照文档安装相同的包库环境。

pip freeze > requirements.txt # 提取当前环境包库信息，行程requirements.txt
pip install -r requirements.txt # 按照requirments.txt安装包库