python的包管理工具_Python的包管理工具

Python的包管理工具

作者：尹正杰

版权声明：原创作品，谢绝转载！否则将追究法律责任。

一.为什么使用包管理

Python的模块或者源文件直接可以复制到目标项目目录中，就可以导入使用了。 但是为了更多项目调用使用，或者共享给别人，就需要打包，或发布到网络，以便供人使用。 目的也是为了复用。

Pypi(Python Package Index)，公共的模块存储中心，https://pypi.python.org/pypi

二.包管理的常见工具

1>.distutils

官方库distutils，使用安装脚本setup.py 来构建、安装包。

从1998年就是标准库的一部分，直到2000年停止开发。

2>.setuptools

它是替代distutils的增强版工具集，包含easy_install工具，使用ez_setup.py文件。支持egg格式的构建和安装。

提供查询、下载、安装、构建、发布、管理等包管理功能。

setuptools是包管理的核心模块。

后来，setuptools开发缓慢了，出现基于setuptools的distribute来替代setuptools。2013年，这两个 项目重新合并，distribute被废弃，setuptools依然是Python安装打包的标准方式。

3>.pip

pip目前包管理的事实标准。

构建在setuptools之上，替代easy_install的。同样提供丰富的包管理功能。

Python3.4之前，需要单独安装，从Python3.4开始直接包含在安装文件中。

4>.wheel

wheel格式定义在PEP427中。

wheel是zip打包的扩展名为.whl的文件，文件中不包含.pyc文件。

提供 bdist_wheel 作为 setuptools 的扩展命令，这个命令可以用来生成新打包格式 wheel。

pip 从1.4版本开始 提供了一个 wheel 子命令来安装 wheel 包。当然，需要先安装 wheel 模块。

它可以让Python库以二进制形式安装，而不需要在本地编译。

三.使用setup.py打包

1>.编写setup.py文件内容

setup.py创建一个源代码分发包的例子，参照例子 https://docs.python.org/3.5/distutils/setupscript. html。

可以在帮助文档chm上搜索索引setup，点击最上面的Distributing Python Modules (Legacy version) ，然后选择 Writing the Setup Script。

包结构即代码如下图所示。

1 #！/usr/bin/env python

2 #_*_conding:utf-8_*_

3 #@author :yinzhengjie

4 #blog:http://www.cnblogs.com/yinzhengjie

5

6 from distutils.core importsetup7

8 """

9

10 导入setup函数并传参11 name名字12 version 版本13 packages=[] 打包列表，14 packages=['hadoop','hadoop.hdfs','hadoop.mapreduce','hadoop.mapreduce.yarn'] 逐级建立目录进行打包15 description 描述信息16 author 作者17 author_email 作者邮件18 url 包的主页，可以不写19 """

20 setup(name='hadoop',21 version='v1.0',22 packages=['hadoop','hadoop.hdfs','hadoop.mapreduce','hadoop.mapreduce.yarn'],23 description='Python test hadoop',24 author='Jason YIN',25 author_email='y1053419035@qq.com',26 url='https://www.cnblogs.com/yinzhengjie/tag/Python%E5%9F%BA%E7%A1%80%E7%9F%A5%E8%AF%86/',27 )

setup.py文件内容

2>.查询命令的帮助信息

(venv) JasonYin最强王者:08.模块化 yinzhengjie$ pwd/yinzhengjie/python/devops/python基础/08.模块化

(venv) JasonYin最强王者:08.模块化 yinzhengjie$

(venv) JasonYin最强王者:08.模块化 yinzhengjie$ python setup.py --help-commands

Standard commands:

build build everything needed to install

build_py"build"pure Python modules (copy to build directory)

build_ext build C/C++ extensions (compile/link to build directory)

build_clib build C/C++libraries used by Python extensions

build_scripts"build" scripts (copy and fixup #! line)

clean clean up temporary files from 'build'command

install install everythingfrombuild directory

install_lib install all Python modules (extensionsandpure Python)

install_headers install C/C++header files

install_scripts install scripts (Pythonorotherwise)

install_data install data files

sdist create a source distribution (tarball, zip file, etc.)

register register the distribution with the Python package index

bdist create a built (binary) distribution

bdist_dumb create a"dumb"built distribution

bdist_rpm create an RPM distribution

bdist_wininst create an executable installerforMS Windows

check perform some checks on the package

upload upload binary package to PyPI

usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]or: setup.py --help [cmd1 cmd2 ...]or: setup.py --help-commandsor: setup.py cmd --help

(venv) JasonYin最强王者:08.模块化 yinzhengjie$

3>.build命令，编译

(venv) JasonYin最强王者:08.模块化 yinzhengjie$ ll

total8drwxr-xr-x 2 yinzhengjie wheel 68 Nov 15 20:34 __pycache__drwxr-xr-x 5 yinzhengjie wheel 170 Nov 15 20:34hadoop-rw-r--r-- 1 yinzhengjie wheel 804 Nov 15 20:33setup.py

(venv) JasonYin最强王者:08.模块化 yinzhengjie$

(venv) JasonYin最强王者:08.模块化 yinzhengjie$ python setup.py build #创建一个build目录并将打包的内容放在里面。

running build

running build_py

creating build

creating build/lib

creating build/lib/hadoop

copying hadoop/__init__.py -> build/lib/hadoop

creating build/lib/hadoop/hdfs

copying hadoop/hdfs/__init__.py -> build/lib/hadoop/hdfs

copying hadoop/hdfs/datanode.py -> build/lib/hadoop/hdfs

copying hadoop/hdfs/namenode.py -> build/lib/hadoop/hdfs

creating build/lib/hadoop/mapreduce

copying hadoop/mapreduce/__init__.py -> build/lib/hadoop/mapreduce

creating build/lib/hadoop/mapreduce/yarn

copying hadoop/mapreduce/yarn/__init__.py -> build/lib/hadoop/mapreduce/yarn

(venv) JasonYin最强王者:08.模块化 yinzhengjie$

(venv) JasonYin最强王者:08.模块化 yinzhengjie$ ll

total8drwxr-xr-x 2 yinzhengjie wheel 68 Nov 15 20:34 __pycache__drwxr-xr-x 3 yinzhengjie wheel 102 Nov 15 20:45build

drwxr-xr-x 5 yinzhengjie wheel 170 Nov 15 20:34hadoop-rw-r--r-- 1 yinzhengjie wheel 804 Nov 15 20:33setup.py

(venv) JasonYin最强王者:08.模块化 yinzhengjie$

(venv) JasonYin最强王者:08.模块化 yinzhengjie$ ll -R build

total 0

drwxr-xr-x 3 yinzhengjie wheel 102 Nov 15 20:45lib

build/lib:

total 0

drwxr-xr-x 5 yinzhengjie wheel 170 Nov 15 20:45hadoop

build/lib/hadoop:

total8

-rw-r--r-- 1 yinzhengjie wheel 127 Nov 15 19:29 __init__.py

drwxr-xr-x 5 yinzhengjie wheel 170 Nov 15 20:45hdfs

drwxr-xr-x 4 yinzhengjie wheel 136 Nov 15 20:45mapreduce

build/lib/hadoop/hdfs:

total24

-rw-r--r-- 1 yinzhengjie wheel 132 Nov 15 19:29 __init__.py-rw-r--r-- 1 yinzhengjie wheel 142 Nov 15 19:32datanode.py-rw-r--r-- 1 yinzhengjie wheel 238 Nov 15 19:35namenode.py

build/lib/hadoop/mapreduce:

total8

-rw-r--r-- 1 yinzhengjie wheel 132 Nov 15 19:29 __init__.py

drwxr-xr-x 3 yinzhengjie wheel 102 Nov 15 20:45yarn

build/lib/hadoop/mapreduce/yarn:

total8

-rw-r--r-- 1 yinzhengjie wheel 107 Nov 15 19:26 __init__.py

(venv) JasonYin最强王者:08.模块化 yinzhengjie$

(venv) JasonYin最强王者:08.模块化 yinzhengjie$ ll -R build

4>.install命令，安装 (将安装包安装到python的“site-packages”)

(venv) JasonYin最强王者:08.模块化 yinzhengjie$ python setup.py install 　　 #build后就可以install，直接运行,如果没有build，会先build编译，然后安装。

running install

running build

running build_py

running install_lib

creating/yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop

copying build/lib/hadoop/__init__.py -> /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop

creating/yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/hdfs

copying build/lib/hadoop/hdfs/__init__.py -> /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/hdfs

copying build/lib/hadoop/hdfs/datanode.py -> /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/hdfs

copying build/lib/hadoop/hdfs/namenode.py -> /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/hdfs

creating/yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/mapreduce

copying build/lib/hadoop/mapreduce/__init__.py -> /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/mapreduce

creating/yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/mapreduce/yarn

copying build/lib/hadoop/mapreduce/yarn/__init__.py -> /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/mapreduce/yarn

byte-compiling /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/__init__.py to __init__.cpython-37.pyc

byte-compiling /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/hdfs/__init__.py to __init__.cpython-37.pyc

byte-compiling /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/hdfs/datanode.py to datanode.cpython-37.pyc

byte-compiling /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/hdfs/namenode.py to namenode.cpython-37.pyc

byte-compiling /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/mapreduce/__init__.py to __init__.cpython-37.pyc

byte-compiling /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/mapreduce/yarn/__init__.py to __init__.cpython-37.pyc

running install_egg_info

Writing/yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop-v1.0-py3.7.egg-info

(venv) JasonYin最强王者:08.模块化 yinzhengjie$

5>.sdist命令

(venv) JasonYin最强王者:08.模块化 yinzhengjie$ ll

total8drwxr-xr-x 2 yinzhengjie wheel 68 Nov 15 20:34__pycache__

drwxr-xr-x 3 yinzhengjie wheel 102 Nov 15 20:45build

drwxr-xr-x 5 yinzhengjie wheel 170 Nov 15 20:34hadoop-rw-r--r-- 1 yinzhengjie wheel 804 Nov 15 20:33setup.py

(venv) JasonYin最强王者:08.模块化 yinzhengjie$

(venv) JasonYin最强王者:08.模块化 yinzhengjie$ python setup.py sdist #创建源代码的分发包,产生一个dist目录，里面生成一个带版本号的压缩包。

running sdist

running check

warning: sdist: manifest template'MANIFEST.in' does not exist (using default filelist)

warning: sdist: standardfilenot found: should have one of README, README.txt, README.rst

writing manifestfile 'MANIFEST'creating hadoop-v1.0creating hadoop-v1.0/hadoop

creating hadoop-v1.0/hadoop/hdfs

creating hadoop-v1.0/hadoop/mapreduce

creating hadoop-v1.0/hadoop/mapreduce/yarn

making hard linksin hadoop-v1.0...

hard linking setup.py-> hadoop-v1.0hard linking hadoop/__init__.py -> hadoop-v1.0/hadoop

hard linking hadoop/hdfs/__init__.py -> hadoop-v1.0/hadoop/hdfs

hard linking hadoop/hdfs/datanode.py -> hadoop-v1.0/hadoop/hdfs

hard linking hadoop/hdfs/namenode.py -> hadoop-v1.0/hadoop/hdfs

hard linking hadoop/mapreduce/__init__.py -> hadoop-v1.0/hadoop/mapreduce

hard linking hadoop/mapreduce/yarn/__init__.py -> hadoop-v1.0/hadoop/mapreduce/yarn

creating dist

Creatingtararchive

removing'hadoop-v1.0'(and everything under it)

(venv) JasonYin最强王者:08.模块化 yinzhengjie$

(venv) JasonYin最强王者:08.模块化 yinzhengjie$ ll

total16

-rw-r--r-- 1 yinzhengjie wheel 206 Nov 17 10:02MANIFEST

drwxr-xr-x 2 yinzhengjie wheel 68 Nov 15 20:34__pycache__

drwxr-xr-x 3 yinzhengjie wheel 102 Nov 15 20:45build

drwxr-xr-x 3 yinzhengjie wheel 102 Nov 17 10:02dist

drwxr-xr-x 5 yinzhengjie wheel 170 Nov 15 20:34hadoop-rw-r--r-- 1 yinzhengjie wheel 804 Nov 15 20:33setup.py

(venv) JasonYin最强王者:08.模块化 yinzhengjie$

(venv) JasonYin最强王者:08.模块化 yinzhengjie$ catMANIFEST

#file GENERATED by distutils, doNOT edit

setup.py

hadoop/__init__.py

hadoop/hdfs/__init__.py

hadoop/hdfs/datanode.py

hadoop/hdfs/namenode.py

hadoop/mapreduce/__init__.py

hadoop/mapreduce/yarn/__init__.py

(venv) JasonYin最强王者:08.模块化 yinzhengjie$

(venv) JasonYin最强王者:08.模块化 yinzhengjie$ ll dist/ #在其他地方解压缩这个文件，里面有setup.py，就可以使用"python setup.py install"安装了， 也可以"pip hadoop-v1.0.tar.gz"直接使用pip安装这个压缩包。

total8

-rw-r--r-- 1 yinzhengjie wheel 1151 Nov 17 10:02 hadoop-v1.0.tar.gz

(venv) JasonYin最强王者:08.模块化 yinzhengjie$

(venv) JasonYin最强王者:08.模块化 yinzhengjie$

6>.bdist命令

二进制分发包，或称作安装程序。它可以生成目标操作系统的安装程序。

可以把自己写好的模块发布到公共的Pypi上，也可以搭建Pypi私服，供企业内部使用。

注意:

Pypi里面的模块没有太好的审核机制，不保证安全，请慎重使用。

制作windows下的安装包

$ python setup.py bdist_wininst

$ python setup.py bdist_msi

$ python setup.py bdist--format=msi

rpm包

$ python setup.py bdist_rpm

$ python setup.py bdist--format=rpm

压缩文件

$ python setup.py bdist--format=zip$ python setup.py bdist--format=gztar

四.wheel包

1>.需要安装依赖包

(venv) JasonYin最强王者:08.模块化 yinzhengjie$ pip installwheel

Collecting wheel

Downloading https://files.pythonhosted.org/packages/00/83/b4a77d044e78ad1a45610eb88f745be2fd2c6d658f9798a15e384b7d57c9/wheel-0.33.6-py2.py3-none-any.whl

Installing collected packages: wheel

Successfully installed wheel-0.33.6(venv) JasonYin最强王者:08.模块化 yinzhengjie$

2>.相关命令

$ python setup.py bdist_egg

$ python setup.py bdist_wheel