python汇 包管理

1 包管理

2 包管理工具

相关术语：

https://docs.python.org/2/distutils/introduction.html

https://python-packaging-user-guide.readthedocs.org/en/latest/glossary.html

2.0 工具使用建议

2.0.1 安装

1 使用pip从pypi安装包, 使用easy_install安装eggs

2 使用virtualenv或pyvenv创建独立的虚拟Python环境

3 使用pip wheel以便后期安装

4 cross-platform software stacks：buildout, Hashdist or conda

2.0.2 打包

1 使用setuptools定义工程，创建source distributions

2 使用setuptools的bdist_wheel扩展创建wheel

3 使用twine上传版本到pypi

参考: https://python-packaging-user-guide.readthedocs.org/en/latest/current.html

2.1 distutils

distutils用于发布Python应用程序，没有提供定义其它依赖包的功能

2.1.1 Distributing Python Modules

2.1.1.1 打包流程

1 写setup.py脚本

2 （可选）写setup.cfg配置文件

3 创建source distribution

4 （可选）创建一个或多个built (binary) distributions

2.1.1.2 setup.py

The project specification file for distutils and setuptools.  

2.1.1.3 setup.cfg

2.1.1.4 创建source distributions

1 MANIFEST.in文件配置

 

2.1.1.5 创建built distributions

2.1.2 Installing Python Modules

$python setup.py install

参考：
https://www.python.org/community/sigs/current/distutils-sig/doc/
https://docs.python.org/2/distutils/
https://docs.python.org/2/install/

2.2 setuptools

2.2.1 setuptools

增强distutils的功能并简化setup.py脚本，
    最大的优势：
        1 增强包管理能力
        2 创建和发布egg包，特别是对其它包具有依赖性的状况
    要求：Python 2.6 or later.
    安装：下载并运行引导程序ez_setup.py

        wget https://bootstrap.pypa.io/ez_setup.py -O - | sudo python
        curl https://bootstrap.pypa.io/ez_setup.py -o - | python

        
        Debian/Ubuntu下可以直接使用apt安装：
        $ sudo apt-get install python-setuptools
        
    更新：
        $ sudo python ez_setup.py -U setuptools

2.2.2 easy_install

setuptools包里的一个模块，python默认的包管理工具
    安装：安装setuptools后，easy_install被自动复制到bin目录，如/usr/local/bin/easy_install


    安装包：默认从internet下载文件
    从PYPI下载并自动安装：easy_install SQLObject
    从URL下载并自动安装：easy_install http://example.com/path/to/MyPackage-1.2.3.tgz
    安装下载好的egg文件: easy_install /my_downloads/OtherPackage-3.2.1-py2.3.egg
    easy_install .
    更新包：
    easy_install "SomePackage==2.0"
    easy_install "SomePackage>2.0"
    easy_install --upgrade SomePackage

2.2.3 egg

"Eggs are to Pythons as Jars are to Java..."，一种单文件的可导入的发布格式
    egg文件用setuptools打包    

2.2.3.1 两种格式：

    1 .egg format: a directory or zipfile containing: 
        1 the project's code and resources, 
        2 an EGG-INFO subdirectory that contains the project's metadata
    2 .egg-info format: a file or directory placed adjacent to the project's code and resources, that directly contains the project's metadata.

2.2.3.2 

参考：        
http://peak.telecommunity.com/DevCenter/setuptools
http://peak.telecommunity.com/DevCenter/PythonEggs
http://peak.telecommunity.com/DevCenter/EggFormats
https://pypi.python.org/pypi/setuptools
http://pythonhosted.org//setuptools/
http://www.ibm.com/developerworks/cn/linux/l-cppeak3.html

2.3 pip

2.3.1 安装

    安装：安装依赖于setuptools,必须先安装easy_install，
        方法1：
        $ easy_install pip
        方法2：
        1 curl -O https://bootstrap.pypa.io/get-pip.py
        2 python get-pip.py
        get-pip.py会自动检索并安装easy_install

2.3.2 requirements文件

pip install命令的参数列表， 每个item一行。

2.3.2.1 格式

<requirement specifier>
<archive url/path>
[-e] <local project path>
[-e] <vcs project url>

The syntax of a requirement specifier can be defined in EBNF（扩展巴科斯范式(Extended Backus Naur Form)） as follows:

requirement  ::= project_name versionspec? extras?
versionspec  ::= comparison version (',' comparison version)*
comparison   ::= '<' | '<=' | '!=' | '==' | '>=' | '>' | '~=' | '==='
extras       ::= '[' extralist? ']'
extralist    ::= identifier (',' identifier)*
project_name ::= identifier
identifier   ::= [-A-Za-z0-9_]+
version      ::= [-A-Za-z0-9_.]+

来源：https://pythonhosted.org/setuptools/pkg_resources.html#requirement-objects
还支持Package Index Options：

•-i, –index-url
•–extra-index-url
•–no-index
•-f, –find-links
•–allow-external
•–allow-all-external
•–allow-unverified
•–no-use-wheel

注：可以引用另一个requirment文件

在requirements.txt文件中列出项目所需要的包。每个包占一行，通常包含版本号。

$ cat requirements.txt
# The order of packages is significant, because pip processes them in the order
# of appearance. Changing the order has an impact on the overall integration
# process, which may cause wedges in the gate later.

pbr>=0.6,!=0.7,<1.0
python-novaclient>=2.18.0

2.3.2.2 使用

1 保存pip freeze的输出，以重复安装

$ env1/bin/pip freeze > requirements.txt
$ env2/bin/<span style="color:#ff0000;">pip install -r requirements.txt</span>    #在另一个env中安装

2 指定版本号

注意：

pip使用install_requires metadata决定包依赖，不是requirments文件。

参考：

https://pip.pypa.io/en/latest/user_guide.html#requirements-files

https://pip.pypa.io/en/latest/reference/pip_install.html#requirements-file-format

2.3.3 requirements与setup.py比较

待总结

来源：https://caremad.io/2013/07/setup-vs-requirement/

2.3.4 pip与easy_install比较

	pip	easy_install
Installs from Wheels	Yes	No
Uninstall Packages	Yes (pip uninstall)	No
Dependency Overrides	Yes (Requirements Files)	No
List Installed Packages	Yes (pip list and pip freeze)	No
PEP438 Support	Yes	No
Installation format	‘Flat’ packages with egg-info metadata.	Encapsulated Egg format
sys.path modification	No	Yes
Installs from Eggs	No	Yes
pylauncher support	No	Yes [1]
Multi-version Installs	No	Yes

来源：https://python-packaging-user-guide.readthedocs.org/en/latest/pip_easy_install.html#pip-vs-easy-install

2.3.5 wheel

wheel是一种Built Distribution格式，基于egg， 被pip支持以代替egg

wheel与egg区别

•Wheel has an official PEP. Egg did not.
•Wheel is a distribution format, i.e a packaging format. [1] Egg was both a distribution format and a runtime installation format (if left zipped), and was designed to be importable.
•Wheel archives do not include .pyc files. Therefore, when the distribution only contains python files (i.e. no compiled extensions), and is compatible with Python 2 and 3, it’s possible for a wheel to be “universal”, similar to an sdist.
•Wheel uses PEP376-compliant .dist-info directories. Egg used .egg-info.
•Wheel has a richer file naming convention. A single wheel archive can indicate its compatibility with a number of Python language versions and implementations, ABIs, and system architectures.
•Wheel is versioned. Every wheel file contains the version of the wheel specification and the implementation that packaged it.
•Wheel is internally organized by sysconfig path type, therefore making it easier to convert to other formats.

参考：

https://pip.pypa.io/en/latest/ 

2.4 packaging(distutils2)

参考：

https://python-packaging-user-guide.readthedocs.org/en/latest/index.html