arm平台安装onnx出现的问题记录

最新推荐文章于 2024-05-09 19:22:57 发布

走错路的程序员

最新推荐文章于 2024-05-09 19:22:57 发布

阅读量4k

点赞数 3

分类专栏：机器视觉 ubuntu 神经网络

本文链接：https://blog.csdn.net/phker/article/details/118207354

版权

机器视觉同时被 3 个专栏收录

40 篇文章 8 订阅

订阅专栏

ubuntu

7 篇文章 1 订阅

订阅专栏

神经网络

3 篇文章 0 订阅

订阅专栏

arm平台安装onnx==1.6.0 报错如下.

	  CMake Warning at CMakeLists.txt:394 (find_package):
      By not providing "Findpybind11.cmake" in CMAKE_MODULE_PATH this project has
      asked CMake to find a package configuration file provided by "pybind11",
      but CMake did not find one.
    
      Could not find a package configuration file provided by "pybind11"
      (requested version 2.2) with any of the following names:
    
        pybind11Config.cmake
        pybind11-config.cmake
    
      Add the installation prefix of "pybind11" to CMAKE_PREFIX_PATH or set
      "pybind11_DIR" to a directory containing one of the above files.  If
      "pybind11" provides a separate development package or SDK, be sure it has
      been installed.
      /// 下面的不是重点. 错误在上面.
	--
    -- ******** Summary ********
    --   CMake version         : 3.10.2
    --   CMake command         : /usr/bin/cmake
    --   System                : Linux
    --   C++ compiler          : /usr/bin/c++
    --   C++ compiler version  : 7.5.0
    --   CXX flags             :  -Wnon-virtual-dtor
    --   Build type            : Release
    --   Compile definitions   :
    --   CMAKE_PREFIX_PATH     :
    --   CMAKE_INSTALL_PREFIX  : /usr/local
    --   CMAKE_MODULE_PATH     :
    --
    --   ONNX version          : 1.6.0
    --   ONNX NAMESPACE        : onnx
    --   ONNX_BUILD_TESTS      : OFF
    --   ONNX_BUILD_BENCHMARKS : OFF
    --   ONNX_USE_LITE_PROTO   : OFF
    --   ONNXIFI_DUMMY_BACKEND : OFF
    --   ONNXIFI_ENABLE_EXT    : OFF
    --
    --   Protobuf compiler     : /usr/bin/protoc
    --   Protobuf includes     : /usr/include
    --   Protobuf libraries    : /usr/lib/aarch64-linux-gnu/libprotobuf.so;-lpthread
    --   BUILD_ONNX_PYTHON     : ON
    --     Python version      :
    --     Python executable   : /home/firefly/Desktop/AI1.6.1/vvv/bin/python3.6
    --     Python includes     : /usr/include/python3.6m
    -- Configuring done
    -- Generating done
    -- Build files have been written to: /tmp/pip-install-pc9fr36z/onnx_966c7d1f68e74990b7a6797c009e08b1/.setuptools-cmake-build
    Scanning dependencies of target gen_onnx_proto
    Scanning dependencies of target onnxifi_loader
    Scanning dependencies of target onnxifi_dummy
    [  1%] Running gen_proto.py on onnx/onnx.in.proto
    [  3%] Building C object CMakeFiles/onnxifi_dummy.dir/onnx/onnxifi_dummy.c.o
    [  4%] Building C object CMakeFiles/onnxifi_loader.dir/onnx/onnxifi_loader.c.o
    /tmp/pip-install-pc9fr36z/onnx_966c7d1f68e74990b7a6797c009e08b1/onnx/onnxifi_dummy.c: In function ‘onnxGetExtensionFunctionAddress’:
    /tmp/pip-install-pc9fr36z/onnx_966c7d1f68e74990b7a6797c009e08b1/onnx/onnxifi_dummy.c:173:21: warning: assignment from incompatible pointer type [-Wincompatible-pointer-types]
               *function = &onnxGetExtensionFunctionAddress;
                         ^
    /tmp/pip-install-pc9fr36z/onnx_966c7d1f68e74990b7a6797c009e08b1/onnx/onnxifi_dummy.c:176:21: warning: assignment from incompatible pointer type [-Wincompatible-pointer-types]
               *function = &onnxSetIOAndRunGraph;
                         ^
    [  6%] Linking C static library libonnxifi_loader.a
    Processing /tmp/pip-install-pc9fr36z/onnx_966c7d1f68e74990b7a6797c009e08b1/onnx/onnx.in.proto
    Writing /tmp/pip-install-pc9fr36z/onnx_966c7d1f68e74990b7a6797c009e08b1/.setuptools-cmake-build/onnx/onnx-ml.proto
    Writing /tmp/pip-install-pc9fr36z/onnx_966c7d1f68e74990b7a6797c009e08b1/.setuptools-cmake-build/onnx/onnx-ml.proto3
    generating /tmp/pip-install-pc9fr36z/onnx_966c7d1f68e74990b7a6797c009e08b1/.setuptools-cmake-build/onnx/onnx_pb.py
    [  8%] Running C++ protocol buffer compiler on /tmp/pip-install-pc9fr36z/onnx_966c7d1f68e74990b7a6797c009e08b1/.setuptools-cmake-build/onnx/onnx-ml.proto
    [  9%] Linking C shared library libonnxifi_dummy.so
    [  9%] Built target onnxifi_loader
    Scanning dependencies of target onnxifi_wrapper
    [ 11%] Building C object CMakeFiles/onnxifi_wrapper.dir/onnx/onnxifi_wrapper.c.o
    [ 11%] Built target onnxifi_dummy
    Writing mypy to onnx/onnx_ml_pb2.pyi
    [ 11%] Built target gen_onnx_proto
    [ 13%] Running gen_proto.py on onnx/onnx-operators.in.proto
    Processing /tmp/pip-install-pc9fr36z/onnx_966c7d1f68e74990b7a6797c009e08b1/onnx/onnx-operators.in.proto
    Writing /tmp/pip-install-pc9fr36z/onnx_966c7d1f68e74990b7a6797c009e08b1/.setuptools-cmake-build/onnx/onnx-operators-ml.proto
    Writing /tmp/pip-install-pc9fr36z/onnx_966c7d1f68e74990b7a6797c009e08b1/.setuptools-cmake-build/onnx/onnx-operators-ml.proto3
    generating /tmp/pip-install-pc9fr36z/onnx_966c7d1f68e74990b7a6797c009e08b1/.setuptools-cmake-build/onnx/onnx_operators_pb.py
    [ 14%] Running C++ protocol buffer compiler on /tmp/pip-install-pc9fr36z/onnx_966c7d1f68e74990b7a6797c009e08b1/.setuptools-cmake-build/onnx/onnx-operators-ml.proto
    [ 16%] Linking C shared module libonnxifi.so
    [ 16%] Built target onnxifi_wrapper
    Writing mypy to onnx/onnx_operators_ml_pb2.pyi
    Scanning dependencies of target onnx_proto
    [ 18%] Building CXX object CMakeFiles/onnx_proto.dir/onnx/onnx-operators-ml.pb.cc.o
    [ 19%] Building CXX object CMakeFiles/onnx_proto.dir/onnx/onnx-ml.pb.cc.o
    [ 21%] Linking CXX static library libonnx_proto.a
    [ 24%] Built target onnx_proto
    Scanning dependencies of target onnx
    [ 26%] Building CXX object CMakeFiles/onnx.dir/onnx/common/status.cc.o
    [ 27%] Building CXX object CMakeFiles/onnx.dir/onnx/common/assertions.cc.o
    [ 29%] Building CXX object CMakeFiles/onnx.dir/onnx/common/ir_pb_converter.cc.o
    [ 32%] Building CXX object CMakeFiles/onnx.dir/onnx/common/interned_strings.cc.o
    [ 32%] Building CXX object CMakeFiles/onnx.dir/onnx/checker.cc.o
    [ 34%] Building CXX object CMakeFiles/onnx.dir/onnx/common/model_helpers.cc.o
    [ 36%] Building CXX object CMakeFiles/onnx.dir/onnx/defs/attr_proto_util.cc.o
    [ 37%] Building CXX object CMakeFiles/onnx.dir/onnx/defs/controlflow/defs.cc.o
    [ 40%] Building CXX object CMakeFiles/onnx.dir/onnx/defs/data_type_utils.cc.o
    [ 40%] Building CXX object CMakeFiles/onnx.dir/onnx/defs/controlflow/old.cc.o
    [ 42%] Building CXX object CMakeFiles/onnx.dir/onnx/defs/function.cc.o
    c++: internal compiler error: Killed (program cc1plus)
    Please submit a full bug report,
    with preprocessed source if appropriate.
    See <file:///usr/share/doc/gcc-7/README.Bugs> for instructions.
    CMakeFiles/onnx.dir/build.make:134: recipe for target 'CMakeFiles/onnx.dir/onnx/common/ir_pb_converter.cc.o' failed
    make[2]: *** [CMakeFiles/onnx.dir/onnx/common/ir_pb_converter.cc.o] Error 4
    make[2]: *** Waiting for unfinished jobs....
    CMakeFiles/Makefile2:279: recipe for target 'CMakeFiles/onnx.dir/all' failed
    make[1]: *** [CMakeFiles/onnx.dir/all] Error 2
    Makefile:129: recipe for target 'all' failed
    make: *** [all] Error 2
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-pc9fr36z/onnx_966c7d1f68e74990b7a6797c009e08b1/setup.py", line 334, in <module>
        'backend-test-tools = onnx.backend.test.cmd_tools:main',
      File "/home/firefly/Desktop/AI1.6.1/vvv/lib/python3.6/site-packages/setuptools/__init__.py", line 129, in setup
        return distutils.core.setup(**attrs)
      File "/usr/lib/python3.6/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/usr/lib/python3.6/distutils/dist.py", line 955, in run_commands
        self.run_command(cmd)
      File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/home/firefly/Desktop/AI1.6.1/vvv/lib/python3.6/site-packages/setuptools/command/install.py", line 61, in run
        return orig.install.run(self)
      File "/usr/lib/python3.6/distutils/command/install.py", line 589, in run
        self.run_command('build')
      File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/usr/lib/python3.6/distutils/command/build.py", line 135, in run
        self.run_command(cmd_name)
      File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/tmp/pip-install-pc9fr36z/onnx_966c7d1f68e74990b7a6797c009e08b1/setup.py", line 209, in run
        self.run_command('cmake_build')
      File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/tmp/pip-install-pc9fr36z/onnx_966c7d1f68e74990b7a6797c009e08b1/setup.py", line 203, in run
        subprocess.check_call(build_args)
      File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
        raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command '['/usr/bin/cmake', '--build', '.', '--', '-j', '6']' returned non-zero exit status 2.
    ----------------------------------------
ERROR: Command errored out with exit status 1: /home/firefly/Desktop/AI1.6.1/vvv/bin/python3.6 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-pc9fr36z/onnx_966c7d1f68e74990b7a6797c009e08b1/setup.py'"'"'; __file__='"'"'/tmp/pip-install-pc9fr36z/onnx_966c7d1f68e74990b7a6797c009e08b1/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-39ksjifz/install-record.txt --single-version-externally-managed --compile --install-headers /home/firefly/Desktop/AI1.6.1/vvv/include/site/python3.6/onnx Check the logs for full command output.

这个狗屎问题, 很是麻烦哦. 百度无果, 从输出信息来看应该是编译的时候少了 pybind11 项目. 于是百度了一下,

后来在这里找到了说明
https://www.cnblogs.com/jourluohua/p/10235317.html

原来必须手动编译, 装好了pybind11之后(源码版本编译ONNX必须手动编译pybind11，使用pip install pybind11并不管用)

然后看看 pybind11 如何编译的, 百度到下面的文章

https://www.cnblogs.com/gaowengang/p/9448723.html

内容如下

pybind11 安装

注意：

安装之前，最好把 /usr/bin/python 指向 python3，安装完再调回去。以后主要使用 python3.
安装之前，最好先安装 boost.

Prerequisites:

$ sudo apt-get install python-dev  (or python3-dev)

$ sudo apt-get install cmake

$ sudo pip install -i https://pypi.tuna.tsinghua.edu.cn/simple/  pytest

$ sudo pip install -i https://pypi.tuna.tsinghua.edu.cn/simple/  numpy

$ sudo pip install -i https://pypi.tuna.tsinghua.edu.cn/simple/  scipy

$ sudo pip install -i https://pypi.tuna.tsinghua.edu.cn/simple/  pybind11

Get pybind11, from website: https://github.com/pybind/pybind11 to get the source package.
从这个网站下载 pybind11 源码

Get Eigen, from website: http://eigen.tuxfamily.org/index.php?title=Main_Page to get the source package.
从这个网站下载 Eigen 源码

for the above two source package, uncompoze it, and,

$ cd  that_folder

$ mkdir build

$ cd build

$ cmake ..                 //# 这个.. 是当前目录的意思无需修改 
$ cmake --build . --config Release --target check
$ make check -j 4  (make and check, not necessary)
$ sudo make install  (maybe should disable python3 in /usr/bin/ if you use python2)

执行安装

$ pip instal onnx==1.6.0

仍然报错, 一开始的错误…

再次百度,
在这里找到了答案, 应该是参数 -Dpybind11_DIR 没有加导致的.
https://stackoverflow.com/questions/63254584/how-to-make-cmake-find-pybind11

cmake … -G “Visual Studio 15 2017” -A x64 `
-Dpybind11_DIR=C:/Users/ben.wolfley/Anaconda3/Library/share/cmake/pybind11/

但问题是我这里install onnx 没有地方写命令传参数Dpybind11_DIR 呀…

还应该是 pybind11没装好. 参照下文.
https://www.cnblogs.com/jourluohua/p/10235317.html
重要的是这么一个信息, ## (源码版本编译ONNX必须手动编译pybind11，使用pip install pybind11并不管用)

还有一个重要的信息的是, 使用

pip install pybind11

安装的文件默认会安装到/user/bin下面. 所以不会安装到虚拟环境下. 所以如果是用虚拟环境venv的朋友要注意了. 虚拟环境下必须用源码编译的方式安装 pybind11

于是开启了痛苦的无边无际的make之坑. 超级郁闷的坑呀. 这里真想报个粗口,
(这是谁TMD设计的模式, make来make去, 就是make不成功. 每次make都有一大堆问题…我心中有一万头草泥马奔腾…)

下载pybind11源码

git clone https://github.com/pybind/pybind11.git

如果你需要eigen库的话，还需要先安装eigen，这里我选择的是无eigen的方式

mkdir build
cd build
cmake ..
make -j 4  # 这里有坑, -j 4 是开启4核并行编译.  要根据机器情况用.
sudo make install

这里有个问题，如果你同时存在python2.x和python3.x版本的话，默认选择的是python3.x版本的，但是我现在的整个环境是python2.7.12，所以就必须禁用掉Python3.x，大家都说禁用，其实就是让自动寻找方式找不到python3.x。
采用的方式是在/usr/bin中所有找到的python3.x相关的文件都移动到另一个文件夹内，

执行到的时候开始报错然后开始编译然后报错.

make -j 4

且每次执行的保存内容稍微有些变化, 都是中断在某个.cpp.o文件的时候就崩了.
而且每次执行make 中断的文件不是同一个文件.

(vvv) firefly@firefly:~/Desktop/AI1.6.1/pybind11/build$ make -j 4
[  2%] Building CXX object tests/CMakeFiles/pybind11_cross_module_tests.dir/pybind11_cross_module_tests.cpp.o
[  7%] Built target cross_module_gil_utils
[  9%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_chrono.cpp.o
[ 11%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_builtin_casters.cpp.o
[ 14%] Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_class.cpp.o
c++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-7/README.Bugs> for instructions.
tests/CMakeFiles/pybind11_tests.dir/build.make:230: recipe for target 'tests/CMakeFiles/pybind11_tests.dir/test_class.cpp.o' failed
make[2]: *** [tests/CMakeFiles/pybind11_tests.dir/test_class.cpp.o] Error 4
make[2]: *** Waiting for unfinished jobs....
c++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-7/README.Bugs> for instructions.
tests/CMakeFiles/pybind11_tests.dir/build.make:206: recipe for target 'tests/CMakeFiles/pybind11_tests.dir/test_chrono.cpp.o' failed
make[2]: *** [tests/CMakeFiles/pybind11_tests.dir/test_chrono.cpp.o] Error 4
^Ctests/CMakeFiles/pybind11_tests.dir/build.make:134: recipe for target 'tests/CMakeFiles/pybind11_tests.dir/test_builtin_casters.cpp.o' failed
make[2]: *** [tests/CMakeFiles/pybind11_tests.dir/test_builtin_casters.cpp.o] Interrupt
tests/CMakeFiles/pybind11_cross_module_tests.dir/build.make:62: recipe for target 'tests/CMakeFiles/pybind11_cross_module_tests.dir/pybind11_cross_module_tests.cpp.o' failed
make[2]: *** [tests/CMakeFiles/pybind11_cross_module_tests.dir/pybind11_cross_module_tests.cpp.o] Interrupt
CMakeFiles/Makefile2:129: recipe for target 'tests/CMakeFiles/pybind11_cross_module_tests.dir/all' failed
make[1]: *** [tests/CMakeFiles/pybind11_cross_module_tests.dir/all] Interrupt
CMakeFiles/Makefile2:166: recipe for target 'tests/CMakeFiles/pybind11_tests.dir/all' failed
make[1]: *** [tests/CMakeFiles/pybind11_tests.dir/all] Interrupt
Makefile:129: recipe for target 'all' failed
make: *** [all] Interrupt

单看报错内容, 共同点似乎都是

recipe for target ‘all’ failed
单单是这个报错信息根本找不到解决方案. 于是我猜测可能是内存太小了? 应该是环境问题.
如果是配置问题, 每次报错应该是一样的. 配置错误有迹可循. 如果是配置错误一般情况下一个文件都不会被编译, 而这里却被有可以编译的文件.
大概率说明不是配置问题, 应该是环境问题. 于是我查看了下
make -j 4

这个命令后面两个参数的意思原来 -j 4的意思是开启4核并行编译的意思.
我这个ARM芯片哪有这么多核, 改成make单核编译吧. 改成

make

结果能够编译成的cpp.o文件数量也多了起来.
在这里插入图片描述
先不管它, 多试几次, 看看能不能最后完成编译. 我感觉可能跟内存有关系, 比较RK3399开发板的内存只有2GB.
现在是每次编译

Building CXX object tests/CMakeFiles/pybind11_tests.dir/test_stl_binders.cpp.o

都编译不过去, 应该是内存不够了.
重启下开发板, 打开监控看看内存使用率.
在这里插入图片描述
至此已基本上明白编译失败的两个原因一个是 make 后面的参数要根据自己机器的情况加.

第二个是内存不够. 那怎么办呢? Ubuntu 有虚拟内存, 看看它怎么开.提高一下虚拟内存
加粗样式

1.首先用命令free查看系统内 Swap 分区大小。命令：

$ free -m

在这里插入图片描述
说明我的系统并未开启虚拟内存
下面查看一下硬盘使用情况

$ df -B G

在这里插入图片描述
可以看的出我的 /userdata目录是挂载的mmcblklp8 这个板载闪存, 有10GB, 使用率是70% , 也就是说, 还有3G的可用空间…快满了呀.
再分它2个GB吧.

2.创建一个 Swap 文件。

$ cd /userdata
$ mkdir swap
$ cd swap
$ sudo dd if=/dev/zero of=swapfile bs=1024 count=200000

在上面命令中的 count 即代表swap文件大小。我们可以自己调整。
命令有点慢, 耐心等待…

3.把生成的文件转换成 Swap 文件

sudo mkswap swapfile

激活 Swap 文件：

sudo swapon swapfile

再次查看的结果。

free -m

total used free shared buffers cachedMem: 1002 967 34 0 22 410-/+ buffers/cache: 534 467Swap: 1053 32 1021

添加虚拟内存成功。

最后如果需要卸载这个 swap 文件，可以进入建立的 swap 文件目录。执行下列命令。

sudo swapoff swapfile  # 别乱用,这是卸载.

如果需要下次开机还一直保持这个 swap ，可以sudo -s换到root然后把它写入 /etc/fstab 文件。

swapfilepath swap swap defaults 0 0

然后回到我们的 pybind11目录再次执行make

$ cd ***/pybind11/build/
$ make

在这里插入图片描述
为了确保没问题

$ make clean # 了一下
$ make

在这里插入图片描述
这些pybind11总算是make成功了. 然后再来

$ sudo make install

在这里插入图片描述
输出这个不知是否算是正常了. 看上去是成功了.

然后编译安装onnx 两种方式, 一种是源代码编译, 另外一种是pip安装
pip方式比较简单,

先试试pip方式

pip install onnx==1.6.0

源码编译方式如下:

git clone https://github.com/onnx/onnx.git
cd ./onnx
python setup.py build
sudo python setup.py install

这里很有可能会出现cmake提示 Protobuf compiler not found 的问题，因此需要安装合适的protobuf编译器

sudo apt-get install  libprotobuf-dev protobuf-compiler

竟然成功了… 哇咔咔…
在这里插入图片描述

总结一下, 关键是两个点, 一个是这个make的报错内容实在是不怎么人性化, 也不报个内存溢出. 或者内存不够, 害我百度了半天没结果. 被误导了 …
第二个是, 没有足够细心的观察问题点. 输出的日志. 若是早点看出来每次输出的不同. 说不定会更快的解决这个问题.
第三个是, 对命令不太仔细. 不了解命令行的意思就照抄. 结果把自己坑进去了. make -j 4 后面的-j 是几个并发进程的意思.

走错路的程序员

关注

3
点赞
踩
9

收藏

觉得还不错? 一键收藏
1
评论
arm平台安装onnx出现的问题记录

arm平台安装onnx==1.6.0 报错如下. CMake Warning at CMakeLists.txt:394 (find_package): By not providing "Findpybind11.cmake" in CMAKE_MODULE_PATH this project has asked CMake to find a package configuration file provided by "pybind11", but CMake
复制链接

扫一扫