Python的包管理工具
作者：尹正杰
版权声明：原创作品，谢绝转载！否则将追究法律责任。
一.为什么使用包管理
Python的模块或者源文件直接可以复制到目标项目目录中，就可以导入使用了。 但是为了更多项目调用使用，或者共享给别人，就需要打包，或发布到网络，以便供人使用。 目的也是为了复用。
Pypi(Python Package Index)，公共的模块存储中心，https://pypi.python.org/pypi
二.包管理的常见工具
1>.distutils
官方库distutils，使用安装脚本setup.py 来构建、安装包。
从1998年就是标准库的一部分，直到2000年停止开发。
2>.setuptools
它是替代distutils的增强版工具集，包含easy_install工具，使用ez_setup.py文件。支持egg格式的构建和安装。
提供查询、下载、安装、构建、发布、管理等包管理功能。
setuptools是包管理的核心模块。
后来，setuptools开发缓慢了，出现基于setuptools的distribute来替代setuptools。2013年，这两个 项目重新合并，distribute被废弃，setuptools依然是Python安装打包的标准方式。
3>.pip
pip目前包管理的事实标准。
构建在setuptools之上，替代easy_install的。同样提供丰富的包管理功能。
Python3.4之前，需要单独安装，从Python3.4开始直接包含在安装文件中。
4>.wheel
wheel格式定义在PEP427中。
wheel是zip打包的扩展名为.whl的文件，文件中不包含.pyc文件。
提供 bdist_wheel 作为 setuptools 的扩展命令，这个命令可以用来生成新打包格式 wheel。
pip 从1.4版本开始 提供了一个 wheel 子命令来安装 wheel 包。当然，需要先安装 wheel 模块。
它可以让Python库以二进制形式安装，而不需要在本地编译。
三.使用setup.py打包
1>.编写setup.py文件内容
setup.py创建一个源代码分发包的例子，参照例子 https://docs.python.org/3.5/distutils/setupscript. html。
可以在帮助文档chm上搜索索引setup，点击最上面的Distributing Python Modules (Legacy version) ，然后选择 Writing the Setup Script。
包结构即代码如下图所示。
1 #！/usr/bin/env python 2 #_*_conding:utf-8_*_ 3 #@author :yinzhengjie 4 #blog:http://www.cnblogs.com/yinzhengjie 5 6 from distutils.core import setup 7 8 """ 9 10 导入setup函数并传参 11 name名字 12 version 版本 13 packages=[] 打包列表， 14 packages=['hadoop','hadoop.hdfs','hadoop.mapreduce','hadoop.mapreduce.yarn'] 逐级建立目录进行打包 15 description 描述信息 16 author 作者 17 author_email 作者邮件 18 url 包的主页，可以不写 19 """ 20 setup(name='hadoop', 21 version='v1.0', 22 packages=['hadoop','hadoop.hdfs','hadoop.mapreduce','hadoop.mapreduce.yarn'], 23 description='Python test hadoop', 24 author='Jason YIN', 25 author_email='y1053419035@qq.com', 26 url='https://www.cnblogs.com/yinzhengjie/tag/Python%E5%9F%BA%E7%A1%80%E7%9F%A5%E8%AF%86/', 27 )
2>.查询命令的帮助信息
(venv) JasonYin最强王者:08.模块化 yinzhengjie$ pwd /yinzhengjie/python/devops/python基础/08.模块化 (venv) JasonYin最强王者:08.模块化 yinzhengjie$ (venv) JasonYin最强王者:08.模块化 yinzhengjie$ python setup.py --help-commands Standard commands: build build everything needed to install build_py "build" pure Python modules (copy to build directory) build_ext build C/C++ extensions (compile/link to build directory) build_clib build C/C++ libraries used by Python extensions build_scripts "build" scripts (copy and fixup #! line) clean clean up temporary files from 'build' command install install everything from build directory install_lib install all Python modules (extensions and pure Python) install_headers install C/C++ header files install_scripts install scripts (Python or otherwise) install_data install data files sdist create a source distribution (tarball, zip file, etc.) register register the distribution with the Python package index bdist create a built (binary) distribution bdist_dumb create a "dumb" built distribution bdist_rpm create an RPM distribution bdist_wininst create an executable installer for MS Windows check perform some checks on the package upload upload binary package to PyPI usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...] or: setup.py --help [cmd1 cmd2 ...] or: setup.py --help-commands or: setup.py cmd --help (venv) JasonYin最强王者:08.模块化 yinzhengjie$
3>.build命令，编译
(venv) JasonYin最强王者:08.模块化 yinzhengjie$ ll total 8 drwxr-xr-x 2 yinzhengjie wheel 68 Nov 15 20:34 __pycache__ drwxr-xr-x 5 yinzhengjie wheel 170 Nov 15 20:34 hadoop -rw-r--r-- 1 yinzhengjie wheel 804 Nov 15 20:33 setup.py (venv) JasonYin最强王者:08.模块化 yinzhengjie$ (venv) JasonYin最强王者:08.模块化 yinzhengjie$ python setup.py build #创建一个build目录并将打包的内容放在里面。 running build running build_py creating build creating build/lib creating build/lib/hadoop copying hadoop/__init__.py -> build/lib/hadoop creating build/lib/hadoop/hdfs copying hadoop/hdfs/__init__.py -> build/lib/hadoop/hdfs copying hadoop/hdfs/datanode.py -> build/lib/hadoop/hdfs copying hadoop/hdfs/namenode.py -> build/lib/hadoop/hdfs creating build/lib/hadoop/mapreduce copying hadoop/mapreduce/__init__.py -> build/lib/hadoop/mapreduce creating build/lib/hadoop/mapreduce/yarn copying hadoop/mapreduce/yarn/__init__.py -> build/lib/hadoop/mapreduce/yarn (venv) JasonYin最强王者:08.模块化 yinzhengjie$ (venv) JasonYin最强王者:08.模块化 yinzhengjie$ ll total 8 drwxr-xr-x 2 yinzhengjie wheel 68 Nov 15 20:34 __pycache__ drwxr-xr-x 3 yinzhengjie wheel 102 Nov 15 20:45 build drwxr-xr-x 5 yinzhengjie wheel 170 Nov 15 20:34 hadoop -rw-r--r-- 1 yinzhengjie wheel 804 Nov 15 20:33 setup.py (venv) JasonYin最强王者:08.模块化 yinzhengjie$
(venv) JasonYin最强王者:08.模块化 yinzhengjie$ ll -R build total 0 drwxr-xr-x 3 yinzhengjie wheel 102 Nov 15 20:45 lib build/lib: total 0 drwxr-xr-x 5 yinzhengjie wheel 170 Nov 15 20:45 hadoop build/lib/hadoop: total 8 -rw-r--r-- 1 yinzhengjie wheel 127 Nov 15 19:29 __init__.py drwxr-xr-x 5 yinzhengjie wheel 170 Nov 15 20:45 hdfs drwxr-xr-x 4 yinzhengjie wheel 136 Nov 15 20:45 mapreduce build/lib/hadoop/hdfs: total 24 -rw-r--r-- 1 yinzhengjie wheel 132 Nov 15 19:29 __init__.py -rw-r--r-- 1 yinzhengjie wheel 142 Nov 15 19:32 datanode.py -rw-r--r-- 1 yinzhengjie wheel 238 Nov 15 19:35 namenode.py build/lib/hadoop/mapreduce: total 8 -rw-r--r-- 1 yinzhengjie wheel 132 Nov 15 19:29 __init__.py drwxr-xr-x 3 yinzhengjie wheel 102 Nov 15 20:45 yarn build/lib/hadoop/mapreduce/yarn: total 8 -rw-r--r-- 1 yinzhengjie wheel 107 Nov 15 19:26 __init__.py (venv) JasonYin最强王者:08.模块化 yinzhengjie$
4>.install命令，安装 (将安装包安装到python的“site-packages”)
(venv) JasonYin最强王者:08.模块化 yinzhengjie$ python setup.py install #build后就可以install，直接运行,如果没有build，会先build编译，然后安装。 running install running build running build_py running install_lib creating /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop copying build/lib/hadoop/__init__.py -> /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop creating /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/hdfs copying build/lib/hadoop/hdfs/__init__.py -> /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/hdfs copying build/lib/hadoop/hdfs/datanode.py -> /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/hdfs copying build/lib/hadoop/hdfs/namenode.py -> /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/hdfs creating /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/mapreduce copying build/lib/hadoop/mapreduce/__init__.py -> /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/mapreduce creating /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/mapreduce/yarn copying build/lib/hadoop/mapreduce/yarn/__init__.py -> /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/mapreduce/yarn byte-compiling /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/__init__.py to __init__.cpython-37.pyc byte-compiling /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/hdfs/__init__.py to __init__.cpython-37.pyc byte-compiling /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/hdfs/datanode.py to datanode.cpython-37.pyc byte-compiling /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/hdfs/namenode.py to namenode.cpython-37.pyc byte-compiling /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/mapreduce/__init__.py to __init__.cpython-37.pyc byte-compiling /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop/mapreduce/yarn/__init__.py to __init__.cpython-37.pyc running install_egg_info Writing /yinzhengjie/python/devops/venv/lib/python3.7/site-packages/hadoop-v1.0-py3.7.egg-info (venv) JasonYin最强王者:08.模块化 yinzhengjie$
5>.sdist命令
(venv) JasonYin最强王者:08.模块化 yinzhengjie$ ll total 8 drwxr-xr-x 2 yinzhengjie wheel 68 Nov 15 20:34 __pycache__ drwxr-xr-x 3 yinzhengjie wheel 102 Nov 15 20:45 build drwxr-xr-x 5 yinzhengjie wheel 170 Nov 15 20:34 hadoop -rw-r--r-- 1 yinzhengjie wheel 804 Nov 15 20:33 setup.py (venv) JasonYin最强王者:08.模块化 yinzhengjie$ (venv) JasonYin最强王者:08.模块化 yinzhengjie$ python setup.py sdist #创建源代码的分发包,产生一个dist目录，里面生成一个带版本号的压缩包。 running sdist running check warning: sdist: manifest template 'MANIFEST.in' does not exist (using default file list) warning: sdist: standard file not found: should have one of README, README.txt, README.rst writing manifest file 'MANIFEST' creating hadoop-v1.0 creating hadoop-v1.0/hadoop creating hadoop-v1.0/hadoop/hdfs creating hadoop-v1.0/hadoop/mapreduce creating hadoop-v1.0/hadoop/mapreduce/yarn making hard links in hadoop-v1.0... hard linking setup.py -> hadoop-v1.0 hard linking hadoop/__init__.py -> hadoop-v1.0/hadoop hard linking hadoop/hdfs/__init__.py -> hadoop-v1.0/hadoop/hdfs hard linking hadoop/hdfs/datanode.py -> hadoop-v1.0/hadoop/hdfs hard linking hadoop/hdfs/namenode.py -> hadoop-v1.0/hadoop/hdfs hard linking hadoop/mapreduce/__init__.py -> hadoop-v1.0/hadoop/mapreduce hard linking hadoop/mapreduce/yarn/__init__.py -> hadoop-v1.0/hadoop/mapreduce/yarn creating dist Creating tar archive removing 'hadoop-v1.0' (and everything under it) (venv) JasonYin最强王者:08.模块化 yinzhengjie$ (venv) JasonYin最强王者:08.模块化 yinzhengjie$ ll total 16 -rw-r--r-- 1 yinzhengjie wheel 206 Nov 17 10:02 MANIFEST drwxr-xr-x 2 yinzhengjie wheel 68 Nov 15 20:34 __pycache__ drwxr-xr-x 3 yinzhengjie wheel 102 Nov 15 20:45 build drwxr-xr-x 3 yinzhengjie wheel 102 Nov 17 10:02 dist drwxr-xr-x 5 yinzhengjie wheel 170 Nov 15 20:34 hadoop -rw-r--r-- 1 yinzhengjie wheel 804 Nov 15 20:33 setup.py (venv) JasonYin最强王者:08.模块化 yinzhengjie$ (venv) JasonYin最强王者:08.模块化 yinzhengjie$ cat MANIFEST # file GENERATED by distutils, do NOT edit setup.py hadoop/__init__.py hadoop/hdfs/__init__.py hadoop/hdfs/datanode.py hadoop/hdfs/namenode.py hadoop/mapreduce/__init__.py hadoop/mapreduce/yarn/__init__.py (venv) JasonYin最强王者:08.模块化 yinzhengjie$ (venv) JasonYin最强王者:08.模块化 yinzhengjie$ ll dist/ #在其他地方解压缩这个文件，里面有setup.py，就可以使用"python setup.py install"安装了， 也可以"pip hadoop-v1.0.tar.gz"直接使用pip安装这个压缩包。 total 8 -rw-r--r-- 1 yinzhengjie wheel 1151 Nov 17 10:02 hadoop-v1.0.tar.gz (venv) JasonYin最强王者:08.模块化 yinzhengjie$ (venv) JasonYin最强王者:08.模块化 yinzhengjie$
6>.bdist命令
二进制分发包，或称作安装程序。它可以生成目标操作系统的安装程序。 可以把自己写好的模块发布到公共的Pypi上，也可以搭建Pypi私服，供企业内部使用。 注意: Pypi里面的模块没有太好的审核机制，不保证安全，请慎重使用。 制作windows下的安装包 $ python setup.py bdist_wininst $ python setup.py bdist_msi $ python setup.py bdist --format=msi
rpm包 $ python setup.py bdist_rpm $ python setup.py bdist --format=rpm
压缩文件 $ python setup.py bdist --format=zip $ python setup.py bdist --format=gztar
四.wheel包
1>.需要安装依赖包
(venv) JasonYin最强王者:08.模块化 yinzhengjie$ pip install wheel Collecting wheel Downloading https://files.pythonhosted.org/packages/00/83/b4a77d044e78ad1a45610eb88f745be2fd2c6d658f9798a15e384b7d57c9/wheel-0.33.6-py2.py3-none-any.whl Installing collected packages: wheel Successfully installed wheel-0.33.6 (venv) JasonYin最强王者:08.模块化 yinzhengjie$
2>.相关命令
$ python setup.py bdist_egg
$ python setup.py bdist_wheel