调研了一下打包python程序的方法,可以参考https://www.cnblogs.com/mywolrd/p/4756005.html和https://baijiahao.baidu.com/s?id=1627375537998184265&wfr=spider&for=pc。总结发现,使用pyinstaller打包最方便,而且迁移到其他环境下,不虚再次部署python环境。下面介绍pyinstaller的安装和使用方法。
1.安装
在ubuntu下,如果使用Python环境建议通过配置anaconda来简化配置过程。pyinstaller可以通过pip和源码进行安装,pip安装可以自动搜索适合当前python的环境,并同其他库适配。pip安装指令如下:
pip install pyinstaller
2.pyinstaller打包方法
要打包的主程序在文件夹code/main.py下,pyinstaller打包可以自动索引相关依赖(原则上...)。如要打包的程序目录如下
code
├── folder1
│ ├── x.py
│ └── y.py
└── main.py
那么将程序打包为一个可执行文件的方法如下:
pyinstaller -F code/main.py
一般来说,程序会自动索引到相关依赖,生成可执行文件。此时会生成两个文件夹(build和dist)和main.spec文件(配置文件),可执行文件在dist/main中(与要打包的Python程序同名)。打包有时虽然成功了,但是有时运行时会出现缺少依赖的情况,部分情况处理如下。
3.特殊情况处理
(1)tensorflow
如果程序中使用了tensorflow,如上打包的话,运行时可能会报出如下错误:
tensorflow.python.framework.errors_impl.NotFoundError: tensorflow/contrib/util/tensorflow/contrib/framework/python/ops/_checkpoint_ops.so: cannot open shared object file: No such file or directory
或者:
packages/tensorflow/contrib/util/site-packages/tensorflow/contrib/rnn/python/ops/_gru_ops.so
此时,需要在生成的main.spec文件中,添加如下代码,可见spec是兼容python程序的。其中tensorflow_location指tensorflow的安装位置,像我tensorflow_location='/home/**/anaconda3/lib/python3.5/site-packages/tensorflow'
import os
tensorflow_location = '/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow'
tensorflow_binaries = []
for dir_name, sub_dir_list, fileList in os.walk(tensorflow_location):
for file in fileList:
if file.endswith(".so"):
full_file = dir_name + '/' + file
print(full_file)
tensorflow_binaries.append((full_file, '.'))
然后修改部分spec内容:
a = Analysis(...,
binaries=tensorflow_binaries,
...)
之后修改site-packages/tensorflow/python/framework/load_library.py(相对位置),注意load_op_library中的那段代码在打包时用就行,其他时间要注释掉:
def resource_path(relative_path):
"""Due to pyinstaller changing root dir of project filenames need to be processed in order to open properly
Parameters
----------
relative_path : str
String containing filename path
Returns
-------
path : str
path relative to this file on local computer
"""
import sys
import os
try:
# PyInstaller creates a temp folder and stores path in _MEIPASS
base_path = sys._MEIPASS
except Exception:
base_path = os.path.abspath(".")
path = os.path.join(base_path, relative_path)
return path
def load_op_library(library_filename):
"""Loads a TensorFlow plugin, containing custom ops and kernels.
Pass "library_filename" to a platform-specific mechanism for dynamically
loading a library. The rules for determining the exact location of the
library are platform-specific and are not documented here. When the
library is loaded, ops and kernels registered in the library via the
`REGISTER_*` macros are made available in the TensorFlow process. Note
that ops with the same name as an existing op are rejected and not
registered with the process.
Args:
library_filename: Path to the plugin.
Relative or absolute filesystem path to a dynamic library file.
Returns:
A python module containing the Python wrappers for Ops defined in
the plugin.
Raises:
RuntimeError: when unable to load the library or get the python wrappers.
"""
# REMOVE AFTER PYINSTALLER USE
library_filename = resource_path(library_filename.split('/')[-1])
最后只需要再次生成程序就行:
pyinstaller main.spec
生成的程序在有cuda环境配置时,会使用GPU,默认情况下是CPU版本。
(2) sklearn等库中的问题
如果程序中使用了sklearn库,程序运行可能会报出如下等多种错误:module not found error: no module named 'sklearn.neighbors.quad_tree'、ImportError: No module named typedefs。
此时只要在spec中显示引用这些库就行:
hiddenimports=['cython','sklearn','sklearn.ensemble','sklearn.neighbors.typedefs','sklearn.neighbors.quad_tree','sklearn.tree._utils','scipy._lib.messagestream']
pyinstaller生成可执行文件时间较长,请耐心等待。