【报错EC0010、E10001】离线推理onnx模型转换为om模型

内卷焦虑人士

已于 2024-04-28 14:16:29 修改

阅读量991

点赞数 8

文章标签： om onnx 昇腾

于 2024-03-29 11:32:38 首次发布

本文链接：https://blog.csdn.net/weixin_46398647/article/details/137132201

版权

报错内容-1

# atc --model=mnist.onnx --framework=5 --insert_op_conf=aipp.cfg --output=mnist --soc_version=Ascend910
ATC start working now, please wait for a moment.
...
ATC run failed, Please check the detail log, Try 'atc --help' for more information
EC0010: Failed to import Python module [ModuleNotFoundError: No module named 'numpy'.].
        Solution: Check that all required components are properly installed and the specified Python path matches the Python installation directory. (If the path does not match the directory, run set_env.sh inthe installation package.)
        TraceBack (most recent call last):
        [GraphOpt][InitializeInner][InitTbeFunc] Failed to init tbe.[FUNC:InitializeInner][FILE:tbe_op_store_adapter.cc][LINE:1625]
        [SubGraphOpt][PreCompileOp][InitAdapter] InitializeAdapter adapter [tbe_op_adapter] failed! Ret [4294967295][FUNC:InitializeAdapter][FILE:op_store_adapter_manager.cc][LINE:85]
        [SubGraphOpt][PreCompileOp][Init] Initialize op store adapter failed, OpsStoreName[tbe-custom].[FUNC:Initialize][FILE:op_store_adapter_manager.cc][LINE:126]
        [FusionMngr][Init] Op store adapter manager init failed.[FUNC:Initialize][FILE:fusion_manager.cc][LINE:124]
        PluginManager InvokeAll failed.[FUNC:Initialize][FILE:ops_kernel_manager.cc][LINE:96]
        OpsManager initialize failed.[FUNC:InnerInitialize][FILE:gelib.cc][LINE:235]
        GELib::InnerInitialize failed.[FUNC:Initialize][FILE:gelib.cc][LINE:163]

解决方案-1

因为报错说Failed to import Python module，找不到python路径
所以我设置python环境变量

which python
# /root/anaconda3/envs/sakura/bin/python
export PYTHONPATH=/root/anaconda3/envs/sakura/bin/python

报错内容-2

报错-1解决完成后再次运行

# atc --model=mnist.onnx --framework=5 --insert_op_conf=aipp.cfg --output=mnist --soc_version=Ascend910
ATC start working now, please wait for a moment.
...
ATC run failed, Please check the detail log, Try 'atc --help' for more information
EC0010: Failed to import Python module [ModuleNotFoundError: No module named 'tbe'.].
        Solution: Check that all required components are properly installed and the specified Python path matches the Python installation directory. (If the path does not match the directory, run set_env.sh inthe installation package.)
        TraceBack (most recent call last):
        [GraphOpt][InitializeInner][InitTbeFunc] Failed to init tbe.[FUNC:InitializeInner][FILE:tbe_op_store_adapter.cc][LINE:1625]
        [SubGraphOpt][PreCompileOp][InitAdapter] InitializeAdapter adapter [tbe_op_adapter] failed! Ret [4294967295][FUNC:InitializeAdapter][FILE:op_store_adapter_manager.cc][LINE:85]
        [SubGraphOpt][PreCompileOp][Init] Initialize op store adapter failed, OpsStoreName[tbe-custom].[FUNC:Initialize][FILE:op_store_adapter_manager.cc][LINE:126]
        [FusionMngr][Init] Op store adapter manager init failed.[FUNC:Initialize][FILE:fusion_manager.cc][LINE:124]
        PluginManager InvokeAll failed.[FUNC:Initialize][FILE:ops_kernel_manager.cc][LINE:96]
        OpsManager initialize failed.[FUNC:InnerInitialize][FILE:gelib.cc][LINE:235]
        GELib::InnerInitialize failed.[FUNC:Initialize][FILE:gelib.cc][LINE:163]

解决方案-2

报错说No module named ‘tbe’，但是我pip安装tbe，运行还是报错
应该选择自己安装的ascend-toolkit版本号的包中的tbe
- /usr/local/Ascend/ascend-toolkit/8.0.RC1.alpha001有效
- /usr/local/Ascend/ascend-toolkit/latest无效
将tbe复制到虚拟环境site-packages中

cp -r /usr/local/Ascend/ascend-toolkit/8.0.RC1.alpha001/python/site-packages/tbe/ /root/anaconda3/envs/sakura/lib/python3.9/site-packages/tbe

接下来设置路径，参考mindspore官网的手动安装部分-配置环境变量
设置后发现官方给到的路径是错误的
和之前一样使用latest版本是无效的，并且/ascend-toolkit/latest/opp/op_impl/built-in/ai_core/tbe不存在
正确方式参考我下面

TBE operator implementation tool path

export TBE_IMPL_PATH=/usr/local/Ascend/ascend-toolkit/8.0.RC1.alpha001/opp/built-in/op_impl/ai_core/tbe

OPP path

export ASCEND_OPP_PATH==/usr/local/Ascend/ascend-toolkit/8.0.RC1.alpha001/opp

AICPU path

export ASCEND_AICPU_PATH=/usr/local/Ascend/ascend-toolkit/8.0.RC1.alpha001

TBE operator compilation tool path

export PATH=/usr/local/Ascend/ascend-toolkit/8.0.RC1.alpha001/compiler/bin/:${PATH}

Python library that TBE implementation depends on

export PYTHONPATH=${TBE_IMPL_PATH}:${PYTHONPATH}

报错内容-3

报错-2解决完成后再次运行

# atc --model=mnist.onnx --framework=5 --insert_op_conf=aipp.cfg --output=mnist --soc_version=Ascend910
ATC start working now, please wait for a moment.
...
ATC run failed, Please check the detail log, Try 'atc --help' for more information
E10001: Value [linux] for parameter [--host_env_os] is invalid. Reason: os not supported, supported settings are OS types of opp package, support list:
        Solution: Try again with a valid argument.

看了昇腾论坛上很多帖子，只有一个人解决了，就是换了python版本为3.9.5（我是3.9.2），我尝试了，失败

报错疑问-3

在/usr/local/Ascend/ascend-toolkit/set_env.sh配置环境变量时指定了

export ASCEND_TOOLKIT_HOME=/usr/local/Ascend/ascend-toolkit/latest
export PYTHONPATH=${ASCEND_TOOLKIT_HOME}/python/site-packages:${ASCEND_TOOLKIT_HOME}/opp/built-in/op_impl/ai_core/tbe:$PYTHONPATH

这就导致了第一个报错，因为这个PYTHONPATH下就是没有numpy
因为我使用了conda所以不考虑用昇腾的虚拟环境
有些人喜欢直接用python虚拟环境的可以直接把${ASCEND_TOOLKIT_HOME}/python/变成自己的虚拟环境使用

解决方案-3

查看了《ATC模型转换指南》
原来的确要设置Python环境变量，但是设置方法和我上面的不一样

以安装用户在任意目录下执行vi ~/.bashrc，在该文件最后添加下述内容

# 我的虚拟环境名称是sakura
export PATH=/root/anaconda3/envs/sakura/bin:$PATH
export LD_LIBRARY_PATH=/root/anaconda3/envs/sakura/lib:$LD_LIBRARY_PATH

执行:wq!命令保存文件并退出。
执行source ~/.bashrc使环境变量生效。

接下来重试模型转换

(sakura) [root@localhost model]# atc --model=mnist.onnx --framework=5 --insert_op_conf=aipp.cfg --output=mnist --soc_version=Ascend910
ATC start working now, please wait for a moment.
...
ATC run success, welcome to the next use.

几秒钟后，大功告成