海思NNIE开发·yolov5

官方预训练模型转换

1. 下载yolov5-6.0分支源码解压到本地，并配置基础运行环境。
2. 下载官方预训练模型

• yolov5n.pt
• yolov5s.pt
• yolov5m.pt
• …

3. 进入yolov5-6.0目录下，新建文件夹weights，并将步骤2中下载的权重文件放进去。
4. 修改models/yolo.py文件

    def forward(self, x):
        z = []  # inference output
        for i in range(self.nl):
            x[i] = self.m[i](x[i]).sigmoid()  # conv
        #     bs, _, ny, nx = x[i].shape  # x(bs,255,20,20) to x(bs,3,20,20,85)
        #     x[i] = x[i].view(bs, self.na, self.no, ny, nx).permute(0, 1, 3, 4, 2).contiguous()

        #     if not self.training:  # inference
        #         if self.grid[i].shape[2:4] != x[i].shape[2:4] or self.onnx_dynamic:
        #             self.grid[i], self.anchor_grid[i] = self._make_grid(nx, ny, i)

        #         y = x[i].sigmoid()
        #         if self.inplace:
        #             y[..., 0:2] = (y[..., 0:2] * 2. - 0.5 + self.grid[i]) * self.stride[i]  # xy
        #             y[..., 2:4] = (y[..., 2:4] * 2) ** 2 * self.anchor_grid[i]  # wh
        #         else:  # for YOLOv5 on AWS Inferentia https://github.com/ultralytics/yolov5/pull/2953
        #             xy = (y[..., 0:2] * 2. - 0.5 + self.grid[i]) * self.stride[i]  # xy
        #             wh = (y[..., 2:4] * 2) ** 2 * self.anchor_grid[i]  # wh
        #             y = torch.cat((xy, wh, y[..., 4:]), -1)
        #         z.append(y.view(bs, -1, self.no))

        # return x if self.training else (torch.cat(z, 1), x)
        return x[0], x[1], x[2]

5. 新建export_nnie.py文件

import os
import torch
import onnx
from onnxsim import simplify
import onnxoptimizer
import argparse
from models.yolo import Detect, Model

if __name__ == '__main__':
    parser = argparse.ArgumentParser()
    parser.add_argument('--weights', type=str, default='./weights/yolov5n.pt', help='initial weights path') 

    #================================================================
    opt = parser.parse_args()
    print(opt)

    #Save Only weights
    ckpt = torch.load(opt.weights, map_location=torch.device('cpu'))
    torch.save(ckpt['model'].state_dict(), opt.weights.replace(".pt", "-model.pt"))

    #Load model without postprocessing
    new_model = Model("./models/{}.yaml".format(os.path.basename(opt.weights).strip(".pt")))
    new_model.load_state_dict(torch.load(opt.weights.replace(".pt", "-model.pt"), map_location=torch.device('cpu')), False)
    new_model.eval()

    #save to JIT script
    example = torch.rand(1, 3, 640, 640)
    traced_script_module = torch.jit.trace(new_model, example)
    traced_script_module.save(opt.weights.replace(".pt", "-jit.pt"))

    #save to onnx
    f = opt.weights.replace(".pt", ".onnx")
    torch.onnx.export(new_model, example, f, verbose=False, opset_version=12,
                            training=torch.onnx.TrainingMode.EVAL,
                            do_constant_folding=True,
                            input_names=['data'],
                            output_names=['out0','out1','out2'])

    #onnxsim
    model_simp, check = simplify(f)
    assert check, "Simplified ONNX model could not be validated"
    onnx.save(model_simp, opt.weights.replace(".pt", "-sim.onnx"))

    #optimize onnx
    passes = ["extract_constant_to_initializer", "eliminate_unused_initializer"]
    optimized_model = onnxoptimizer.optimize(model_simp, passes)
    onnx.checker.check_model(optimized_model)
    onnx.save(optimized_model, opt.weights.replace(".pt", "-op.onnx"))
    print('finished exporting onnx')

6. 命令行执行python3 export_nnie.py脚本(默认为yolov5n.pt, 加–weights参数可指定权重)，转换成功会输出一下信息, 转换后的模型存于权重同级目录(*-op.onnx后缀模型)

Namespace(weights='./weights/yolov5n.pt')
finished exporting onnx

海思相机植入-模型转换篇

前期准备

• Hi3516/3519芯片系列软件定义相机
• Ruyistudio开发工具 用于转换NNIE模型
• vscode/MobaXterm 用于相机后台调试
• pycaffe环境 支持上采样、permute等自定义操作

详细流程

1. onnx->caffe
   
   可参考yolov5_onnx2caffe开源工程, 转换成功会输出一下信息。
   

2. 打开Ruyistudio开发工具，新建nnie项目, 主要soc version要与目标相机版本相匹配
   

3. 打开.cfg文件，配置onnx2nnie必要参数， 然后点击运行按钮
   

4. 运行成功会输出以下信息
   

海思相机植入-NNIE加载推理篇

• 使用海思NNIE服务接口加载推理模型
  
  

下集预告

• 海思NNIE开发·yolov7