ncnn填坑记录四：pytorch模型转ncnn

上一篇：ncnn填坑记录三：在pc端运行examples
下一篇：ncnn填坑记录五：在安卓端运行ncnn

以torchvision自带的resnent18为例

一.pytorch模型转为onnx

import torch
import torchvision

# An instance of your model
model = torchvision.models.resnet18()

# An example input you would normally provide to your model's forward() method
x = torch.rand(1, 3, 224, 224)

# Export the model
torch_out = torch.onnx.export(model, x, "resnet18.onnx", export_params=True)

运行生成resnet18.onnx

二.简化onnx模型

在将onnx转换为ncnn模型前，我们需要简化onnx模型，以免出现不可编译的情况

安装onnx-smiplifier

pip install onnx-simplifier

简化onnx模型

python -m onnxsim resnet18.onnx resnet18-sim.onnx

三.onnx转换为ncnn

将resnet18-sim.onnx移至ncnn/build/tools/onnx文件夹内，打开终端

./onnx2ncnn resnet18-sim.onnx resnet18.param resnet18.bin

生成resnet18.param和resnet18.bin，移至ncnn/build/examples文件夹内

四.修改相关文件

1.ncnn/examples/CMakeLists.txt

倒数第二行增加：

ncnn_add_example(resnet18)

其余模型可以注释掉

2.resnet18.cpp

在ncnn/examples文件夹内，复制一份squeezenet.cpp，改名为resnet18.cpp并修改内容。

导入权重修改：

    squeezenet.load_param("squeezenet_v1.1.param");
    squeezenet.load_model("squeezenet_v1.1.bin");

    squeezenet.load_param("resnet18.param");
    squeezenet.load_model("resnet18.bin");

输入图片resize大小修改

    ncnn::Mat in = ncnn::Mat::from_pixels_resize(bgr.data, ncnn::Mat::PIXEL_BGR, bgr.cols, bgr.rows, 227, 227);

    ncnn::Mat in = ncnn::Mat::from_pixels_resize(bgr.data, ncnn::Mat::PIXEL_BGR, bgr.cols, bgr.rows, 224, 224);

输入输出修改：

	ex.input("data", in);

    ncnn::Mat out;
    ex.extract("prob", out);

    ex.input("input.1", in);

    ncnn::Mat out;
    ex.extract("Gemm_68", out);

此处的input.1和Gemm_68是打开resnet18.param找到的（可对比squeezenet.param）

五.运行

问题

在ncnn/build下make -j8，并在ncnn/build/examples下./resnet18 test.jpg

(base) lgy@lgy:~/tools/ncnn/build$ make -j8
[  1%] Built target OGLCompiler
[  1%] Built target OSDependent
[  1%] Built target ncnnmerge
[  1%] Built target GenericCodeGen
[  1%] Built target mxnet2ncnn
[  1%] Built target caffe2ncnn
[ 47%] Built target ncnn-generate-spirv
[ 47%] Built target onnx2ncnn
[ 48%] Built target darknet2ncnn
[ 52%] Built target MachineIndependent
[ 52%] Built target glslang
[ 53%] Built target SPIRV
[ 96%] Built target ncnn
Scanning dependencies of target resnet18
[ 96%] Built target ncnn2table
[ 97%] Built target ncnn2mem
[ 98%] Built target ncnnoptimize
[ 98%] Built target ncnn2int8
[ 98%] Building CXX object examples/CMakeFiles/resnet18.dir/resnet18.cpp.o
[ 99%] Built target benchncnn
[100%] Linking CXX executable resnet18
[100%] Built target resnet18
(base) lgy@lgy:~/tools/ncnn/build$ cd examples/
(base) lgy@lgy:~/tools/ncnn/build/examples$ ./resnet18 test.jpg
[0 GeForce GTX 1060 6GB]  queueC=2[8]  queueG=0[16]  queueT=1[2]
[0 GeForce GTX 1060 6GB]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
[0 GeForce GTX 1060 6GB]  fp16-p/s/a=1/1/0  int8-p/s/a=1/1/1
[0 GeForce GTX 1060 6GB]  subgroup=32  basic=1  vote=1  ballot=1  shuffle=1
fopen resnet.bin failed
find_blob_index_by_name Gemm_68 failed
Try
    ex.extract("191", out0);
段错误

方法一

报错参考https://github.com/Tencent/ncnn/issues/868进行修改

把pytorch模型的param和bin文件按使用指北中去除可见字符串的方式得到id.h和mem.h文件，然后在id.h文件就有输入层和输出层的blob名字

(base) lgy@lgy:~/tools/ncnn/build/tools$ ./ncnn2mem resnet18.param resnet18.bin resent18.id.h resnet18.men.h

打开生成的resnet18.id.h

#ifndef NCNN_INCLUDE_GUARD_resent18_id_h
#define NCNN_INCLUDE_GUARD_resent18_id_h
namespace resnet18_param_id {
const int LAYER_input_1 = 0;
const int BLOB_input_1 = 0;
......
const int BLOB_190 = 64;
const int LAYER_Gemm_68 = 57;
const int BLOB_191 = 65;
} // namespace resnet18_param_id
#endif // NCNN_INCLUDE_GUARD_resent18_id_h

输入和输出分别叫BLOB_input_1和BLOB_191
由于ncnn2mem 工具转换出来的是二进制描述文件和内存模型，加载方式变了

squeezenet.load_param_bin("resnet18.param.bin");
squeezenet.load_model("resnet18.bin");

输入输出改为（修改时注意去掉双引号）：

ex.input(resnet18_param_id::BLOB_input_1, in);
ex.extract(resnet18_param_id::BLOB_191, out);

为了使用namespace resnet18_param_id，resnet18.cpp头文件加上

 #include "resnet18.id.h"

方法二

后来参考https://github.com/Tencent/ncnn/issues/2517，发现可以用netron打开resnet18.param


input的name是input.1，output的name是191，而Gemm_68是output的层名。
突然意识最开始报的错就是

find_blob_index_by_name Gemm_68 failed
Try
    ex.extract("191", out0);

把"Gemm_68"改为“191”后，能够正常运行。

(base) lgy@lgy:~/tools/ncnn/build/examples$ ./resnet18 test.jpg 
[0 GeForce GTX 1060 6GB]  queueC=2[8]  queueG=0[16]  queueT=1[2]
[0 GeForce GTX 1060 6GB]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
[0 GeForce GTX 1060 6GB]  fp16-p/s/a=1/1/0  int8-p/s/a=1/1/1
[0 GeForce GTX 1060 6GB]  subgroup=32  basic=1  vote=1  ballot=1  shuffle=1
897 = 214.500000
66 = 182.000000
590 = 171.750000