华北工控EMB3581 瑞芯微Rockchip RK3568，python部署rknn_toolkit_lite2

EMB-3581
瑞芯微Rockchip RK3568 处理器,2LAN,4USB3.0,4USB2.0,7COM,3.5寸板
◆ 支持瑞芯微Rockchip RK3568处理器
◆ 板载4GB LPDDR4内存,最大可支持8GB
◆ 1VGA,1HDMI,1LVDS
◆ 2LAN,4USB3.0,4USB2.0,7COM
◆ 1Mini-PCIe

安装基础环境python3.7（rknn_toolkit_lite2仅支持python3.7，3.9）

网上教程一大堆，这个没啥说的

下载RKNN SDK

https://wiki.t-firefly.com/zh_CN/ROC-RK3568-PC/usage_npu.html
RKNN SDK 为带有 NPU 的 RK3566/RK3568 芯片平台提供编程接口，能够帮助用户部署使用 RKNN-Toolkit2 导出的 RKNN 模型

复制librknn_api支持

运行命令

sudo cp RK_NPU_SDK_1.2.0/release/rknpu2/runtime/RK356X/Linux/librknn_api/aarch64/* /usr/lib

复制

RK_NPU_SDK_1.2.0/release/rknpu2/runtime/RK356X/Linux/librknn_api/aarch64/librknn_api.so
RK_NPU_SDK_1.2.0/release/rknpu2/runtime/RK356X/Linux/librknn_api/aarch64/librknnrt.so

到

/usr/lib

下载运行示例

rknn算子查询1.3

https://github.com/rockchip-linux/rknn-toolkit2

它会报错，问题不大，信息如下
解决方法：把图片换成640×640，或者cv2.resize（640×640）

--> Running model
img shape (224, 224, 3)
E RKNN: [10:37:13.815] rknn_inputs_set, param input size(150528) < model input size(1228800)
E Catch exception when setting inputs.
E Traceback (most recent call last):
  File "/usr/local/python37/lib/python3.7/site-packages/rknnlite/api/rknn_lite.py", line 184, in inference
    self.rknn_runtime.set_inputs(inputs, data_type, data_format, inputs_pass_through=inputs_pass_through)
  File "rknnlite/api/rknn_runtime.py", line 977, in rknnlite.api.rknn_runtime.RKNNRuntime.set_inputs
Exception: Set inputs failed. error code: RKNN_ERR_PARAM_INVALID

output:  None
Traceback (most recent call last):
  File "test.py", line 98, in <module>
    show_top5(outputs)
  File "test.py", line 38, in show_top5
    output = result[0].reshape(-1)
TypeError: 'NoneType' object is not subscriptable

修改后运行结果如下：

模型转换paddle---->onnx------>rknn

paddle---->onnx

前置条件，已导出推理模型

paddle.jit.save(model, args.infer_save_dir)

在aistudio上

!pip uninstall onnx -y
!pip install onnx==1.7
!pip install -U paddle2onnx==0.9.8
!paddle2onnx --model_dir save_model\
            --model_filename infer.pdmodel \
            --params_filename infer.pdiparams \
            --save_file FeaturePyramidNetwork.onnx \
            --opset_version 12 \
            --enable_dev_version True \
            --enable_onnx_checker True \
            --enable_auto_update_opset False

Your model ir_version is higher than the

E load_onnx: Traceback (most recent call last):
E load_onnx:   File "rknn/api/rknn_base.py", line 1182, in rknn.api.rknn_base.RKNNBase.load_onnx
E load_onnx:   File "rknn/api/rknn_base.py", line 603, in rknn.api.rknn_base.RKNNBase._create_ir_and_inputs_meta
E load_onnx:   File "rknn/api/ir_graph.py", line 45, in rknn.api.ir_graph.IRGraph.__init__
E load_onnx:   File "rknn/api/ir_graph.py", line 285, in rknn.api.ir_graph.IRGraph.rebuild
E load_onnx:   File "rknn/api/ir_graph.py", line 149, in rknn.api.ir_graph.IRGraph._clean_model
E load_onnx:   File "rknn/api/ir_graph.py", line 65, in rknn.api.ir_graph.IRGraph.infer_shapes
E load_onnx:   File "/home/LTL/.local/lib/python3.6/site-packages/onnx/checker.py", line 93, in check_model
E load_onnx:     C.check_model(model.SerializeToString())
E load_onnx: onnx.onnx_cpp2py_export.checker.ValidationError: Your model ir_version is higher than the checker's.

我转换的，paddle2onnx命令，最低就只能是V7

需要的V6

解决方法：

!pip uninstall onnx -y
!pip install onnx==1.7

!pip uninstall onnxruntime -y
!pip install onnxruntime==1.6.0

!pip uninstall onnxoptimizer -y
!pip install onnxoptimizer==0.1.0

!pip install -U paddle2onnx==0.9.8

!paddle2onnx    --model_dir=save_model \
    --model_filename infer.pdmodel \
    --params_filename infer.pdiparams \
    --save_file FeaturePyramidNetwork.onnx \
    --opset_version 12 

onnx------>rknn

ImportError: libpython3.6m.so.1.0: cannot open shared object file: No such file or directory

--> Building model
Analysing : 100%|███████████████████████████████████████████████| 165/165 [00:00<00:00, 2812.62it/s]
Quantizating : 100%|█████████████████████████████████████████████| 165/165 [00:00<00:00, 646.19it/s]
E build: Catch exception when building RKNN model!
E build: Traceback (most recent call last):
E build:   File "rknn/api/rknn_base.py", line 1590, in rknn.api.rknn_base.RKNNBase.build
E build:   File "rknn/api/rknn_base.py", line 344, in rknn.api.rknn_base.RKNNBase._generate_rknn
E build:   File "rknn/api/rknn_base.py", line 276, in rknn.api.rknn_base.RKNNBase._export_rknn
E build: ImportError: libpython3.6m.so.1.0: cannot open shared object file: No such file or directory

先看环境中有没有

find / -name libpython3.6m.so.1.0

有则复制

sudo cp /usr/local/python36/lib/libpython3.6m.so.1.0 /usr/local/lib64/
sudo cp /usr/local/python36/lib/libpython3.6m.so.1.0 /usr/lib/ 
sudo cp /usr/local/python36/lib/libpython3.6m.so.1.0 /usr/lib64/

没有就重新编译安装，安装后查看环境中有没有，有则复制

cd Python-3.6.3
./configure --prefix=/usr/local/python36 --enable-shared CFLAGS=-fPIC
make
sudo make install

ImportError: libprotobuf.so.10: cannot open shared object file: No such file or directory

E build: Catch exception when building RKNN model!
E build: Traceback (most recent call last):
E build:   File "rknn/api/rknn_base.py", line 1590, in rknn.api.rknn_base.RKNNBase.build
E build:   File "rknn/api/rknn_base.py", line 344, in rknn.api.rknn_base.RKNNBase._generate_rknn
E build:   File "rknn/api/rknn_base.py", line 276, in rknn.api.rknn_base.RKNNBase._export_rknn
E build: ImportError: libprotobuf.so.10: cannot open shared object file: No such file or directory

解决方法
下载protobufv3.0.0，点击download zip，不要用git clone
下载gmock-1.7.0
版本一定要对齐
把gmock解压出来的目录拷贝到protobuf目录下，改名为gmock，目录结构如下：

cd protobuf 

vim aotogen.sh将删掉以下部分代码，禁用自动下载，已经手动下载好了:

if test ! -e gmock; then
  echo "Google Mock not present.  Fetching gmock-1.7.0 from the web..."
  curl -O https://googlemock.googlecode.com/files/gmock-1.7.0.zip
  unzip -q gmock-1.7.0.zip
  rm gmock-1.7.0.zip
  mv gmock-1.7.0 gmock
fi

安装protobuf，分别执行以下代码

./autogen.sh
./configure
make
make check
make install

配置libprotobuf.conf

sudo vim /etc/ld.so.conf.d/libprotobuf.conf 包含内容：/usr/local/lib
sudo ldconfig 

验证是否安装成功，查看版本号

~$ protoc --version 
~$ libprotoc 3.0.0