前言
香橙派内置了6T算力的NPU，想着可以跑一下yolov5，看看香橙派的速度如何。
在开始部署之前，需要具备一定的linux技能——vim、linux常见指令、conda等等。如果没有这些技能的话，做下去会有一定的难度，可以先看几遍了解一下流程；也可以边学边部署。
香橙派环境
Orangepi5pro 1.0.4 ubuntu jammy desktop xfce linux6.1.43.7z
python=3.10.0
conda:Miniconda3-py310_23.11.0-2-Linux-x86_64.sh
参考文章香橙派5 RK3588 yolov5模型转换rknn及部署踩坑全记录 orangepi 5-CSDN博客
第一步——在PC端生成文件
这一步是在PC端（笔记本或台式电脑）上实现的。
(1)、在电脑上使用WSL安装Ubuntu22.04系统并配置conda环境
可以在应用商城搜索WSL，点击安装
具体可以看这个视频：【WSL2】在你的win10/11电脑上安装Linux子系统+Ubuntu+图形化界面_哔哩哔哩_bilibili
conda环境可以看我的上一篇博客香橙派5pro（orangepi5pro）安装miniconda3_orange pi 5 pro install conda-CSDN博客
虽然是在PC端,但都是linux环境,上传.sh文件安装就行(上传过程自行百度)
(2)下载rknn-toolkit2源码
(3)一、在windows上
①下载yolov5源码以及yolov5s.pt
②将yolov5.pt转换成yolov5.onnx
按照官方文档操作,将class Detect(nn.Moudle)类的子函数forword更改
文件名字
改为:
def forward(self, x): z = [] # inference output for i in range(self.nl): x[i] = self.m[i](x[i]) # conv return x③然后，在yolov5目录下，打开终端输入命令
python export.py --weights yolov5n.pt --data data/coco128.yaml --include onnx --opset 12 --batch-size 1注意注意！opset一定要为12，不然后面onnx转rknn会报错。weights自己选你训练完成的best.pt，data选你自己设置的，这里会生成一个.onnx文件
二、在windows中的linux上
④onnx转化为rknn格式
进入用conda创建的虚拟环境中，假设叫yolo_py310，python=3.10.0
conda activate yolo_py310假设你刚刚将rknn-toolkit2源码和yolov5源码上传到/root/Download/下面
这个rknn是rknn-toolkit2-1.5.2文件的重命名
在rknn/doc/，运行这个指令
pip install -r requirements_cp310-1.5.2.txt -i https://mirror.baidu.com/pypi/simple进入/rknn/packages/中
pip install rknn_toolkit2-1.5.2+b642f30c-cp310-cp310-linux_x86_64.whl
中间会出现这个错误
这个上面这个博客的命令时需要科学上网！！！
完成后，输入命令 python，输入以下代码如果不报错说明安装成功，使用键盘Ctrl+Z退出此模式。
from rknn.api import RKNN
将（2）③中获得的.onnx文件放到rknn/examples/onnx/yolov5文件夹下，终端里进入该文件夹。用你喜欢的编辑器修改 test.py里面的一些内容
根据你的板子情况添加芯片型号
情况像下面的差不多就时成功了
此时会出现一个yolo5s.rknn的文件
⑤准备文件
1.创建一个名为deploy.py的文件
#deploy.py import numpy as np import cv2 from rknnlite.api import RKNNLite RKNN_MODEL = 'yolov5n.rknn' QUANTIZE_ON = True OBJ_THRESH = 0.25 NMS_THRESH = 0.45 IMG_SIZE = 640 CLASSES = ("person", "bicycle", "car", "motorbike ", "aeroplane ", "bus ", "train", "truck ", "boat", "traffic light", "fire hydrant", "stop sign ", "parking meter", "bench", "bird", "cat", "dog ", "horse ", "sheep", "cow", "elephant", "bear", "zebra ", "giraffe", "backpack", "umbrella", "handbag", "tie", "suitcase", "frisbee", "skis", "snowboard", "sports ball", "kite", "baseball bat", "baseball glove", "skateboard", "surfboard", "tennis racket", "bottle", "wine glass", "cup", "fork", "knife ", "spoon", "bowl", "banana", "apple", "sandwich", "orange", "broccoli", "carrot", "hot dog", "pizza ", "donut", "cake", "chair", "sofa", "pottedplant", "bed", "diningtable", "toilet ", "tvmonitor", "laptop ", "mouse ", "remote ", "keyboard ", "cell phone", "microwave ", "oven ", "toaster", "sink", "refrigerator ", "book", "clock", "vase", "scissors ", "teddy bear ", "hair drier", "toothbrush ") def sigmoid(x): return 1 / (1 + np.exp(-x)) def xywh2xyxy(x): # Convert [x, y, w, h] to [x1, y1, x2, y2] y = np.copy(x) y[:, 0] = x[:, 0] - x[:, 2] / 2 # top left x y[:, 1] = x[:, 1] - x[:, 3] / 2 # top left y y[:, 2] = x[:, 0] + x[:, 2] / 2 # bottom right x y[:, 3] = x[:, 1] + x[:, 3] / 2 # bottom right y return y def process(input, mask, anchors): anchors = [anchors[i] for i in mask] grid_h, grid_w = map(int, input.shape[0:2]) box_confidence = sigmoid(input[..., 4]) box_confidence = np.expand_dims(box_confidence, axis=-1) box_class_probs = sigmoid(input[..., 5:]) box_xy = sigmoid(input[..., :2])*2 - 0.5 col = np.tile(np.arange(0, grid_w), grid_w).reshape(-1, grid_w) row = np.tile(np.arange(0, grid_h).reshape(-1, 1), grid_h) col = col.reshape(grid_h, grid_w, 1, 1).repeat(3, axis=-2) row = row.reshape(grid_h, grid_w, 1, 1).repeat(3, axis=-2) grid = np.concatenate((col, row), axis=-1) box_xy += grid box_xy *= int(IMG_SIZE/grid_h) box_wh = pow(sigmoid(input[..., 2:4])*2, 2) box_wh = box_wh * anchors box = np.concatenate((box_xy, box_wh), axis=-1) return box, box_confidence, box_class_probs def filter_boxes(boxes, box_confidences, box_class_probs): """Filter boxes with box threshold. It's a bit different with origin yolov5 post process! # Arguments boxes: ndarray, boxes of objects. box_confidences: ndarray, confidences of objects. box_class_probs: ndarray, class_probs of objects. # Returns boxes: ndarray, filtered boxes. classes: ndarray, classes for boxes. scores: ndarray, scores for boxes. """ boxes = boxes.reshape(-1, 4) box_confidences = box_confidences.reshape(-1) box_class_probs = box_class_probs.reshape(-1, box_class_probs.shape[-1]) _box_pos = np.where(box_confidences >= OBJ_THRESH) boxes = boxes[_box_pos] box_confidences = box_confidences[_box_pos] box_class_probs = box_class_probs[_box_pos] class_max_score = np.max(box_class_probs, axis=-1) classes = np.argmax(box_class_probs, axis=-1) _class_pos = np.where(class_max_score >= OBJ_THRESH) boxes = boxes[_class_pos] classes = classes[_class_pos] scores = (class_max_score* box_confidences)[_class_pos] return boxes, classes, scores def nms_boxes(boxes, scores): """Suppress non-maximal boxes. # Arguments boxes: ndarray, boxes of objects. scores: ndarray, scores of objects. # Returns keep: ndarray, index of effective boxes. """ x = boxes[:, 0] y = boxes[:, 1] w = boxes[:, 2] - boxes[:, 0] h = boxes[:, 3] - boxes[:, 1] areas = w * h order = scores.argsort()[::-1] keep = [] while order.size > 0: i = order[0] keep.append(i) xx1 = np.maximum(x[i], x[order[1:]]) yy1 = np.maximum(y[i], y[order[1:]]) xx2 = np.minimum(x[i] + w[i], x[order[1:]] + w[order[1:]]) yy2 = np.minimum(y[i] + h[i], y[order[1:]] + h[order[1:]]) w1 = np.maximum(0.0, xx2 - xx1 + 0.00001) h1 = np.maximum(0.0, yy2 - yy1 + 0.00001) inter = w1 * h1 ovr = inter / (areas[i] + areas[order[1:]] - inter) inds = np.where(ovr <= NMS_THRESH)[0] order = order[inds + 1] keep = np.array(keep) return keep def yolov5_post_process(input_data): masks = [[0, 1, 2], [3, 4, 5], [6, 7, 8]] anchors = [[10, 13], [16, 30], [33, 23], [30, 61], [62, 45], [59, 119], [116, 90], [156, 198], [373, 326]] boxes, classes, scores = [], [], [] for input, mask in zip(input_data, masks): b, c, s = process(input, mask, anchors) b, c, s = filter_boxes(b, c, s) boxes.append(b) classes.append(c) scores.append(s) boxes = np.concatenate(boxes) boxes = xywh2xyxy(boxes) classes = np.concatenate(classes) scores = np.concatenate(scores) nboxes, nclasses, nscores = [], [], [] for c in set(classes): inds = np.where(classes == c) b = boxes[inds] c = classes[inds] s = scores[inds] keep = nms_boxes(b, s) nboxes.append(b[keep]) nclasses.append(c[keep]) nscores.append(s[keep]) if not nclasses and not nscores: return None, None, None boxes = np.concatenate(nboxes) classes = np.concatenate(nclasses) scores = np.concatenate(nscores) return boxes, classes, scores def draw(image, boxes, scores, classes): """Draw the boxes on the image. # Argument: image: original image. boxes: ndarray, boxes of objects. classes: ndarray, classes of objects. scores: ndarray, scores of objects. all_classes: all classes name. """ for box, score, cl in zip(boxes, scores, classes): top, left, right, bottom = box print('class: {}, score: {}'.format(CLASSES[cl], score)) print('box coordinate left,top,right,down: [{}, {}, {}, {}]'.format(top, left, right, bottom)) top = int(top) left = int(left) right = int(right) bottom = int(bottom) cv2.rectangle(image, (top, left), (right, bottom), (255, 0, 0), 2) cv2.putText(image, '{0} {1:.2f}'.format(CLASSES[cl], score), (top, left - 6), cv2.FONT_HERSHEY_SIMPLEX, 0.6, (0, 0, 255), 2) def letterbox(im, new_shape=(640, 640), color=(0, 0, 0)): # Resize and pad image while meeting stride-multiple constraints shape = im.shape[:2] # current shape [height, width] if isinstance(new_shape, int): new_shape = (new_shape, new_shape) # Scale ratio (new / old) r = min(new_shape[0] / shape[0], new_shape[1] / shape[1]) # Compute padding ratio = r, r # width, height ratios new_unpad = int(round(shape[1] * r)), int(round(shape[0] * r)) dw, dh = new_shape[1] - new_unpad[0], new_shape[0] - new_unpad[1] # wh padding dw /= 2 # divide padding into 2 sides dh /= 2 if shape[::-1] != new_unpad: # resize im = cv2.resize(im, new_unpad, interpolation=cv2.INTER_LINEAR) top, bottom = int(round(dh - 0.1)), int(round(dh + 0.1)) left, right = int(round(dw - 0.1)), int(round(dw + 0.1)) im = cv2.copyMakeBorder(im, top, bottom, left, right, cv2.BORDER_CONSTANT, value=color) # add border return im, ratio, (dw, dh) if __name__ == '__main__': # Create RKNN object rknn = RKNNLite() # load RKNN model print('--> Load RKNN model') ret = rknn.load_rknn(RKNN_MODEL) # Init runtime environment print('--> Init runtime environment') ret = rknn.init_runtime(core_mask=RKNNLite.NPU_CORE_0_1_2) #使用0 1 2三个NPU核心 # ret = rknn.init_runtime('rk3566') if ret != 0: print('Init runtime environment failed!') exit(ret) print('done') # Set inputs img = cv2.imread(IMG_PATH) # img, ratio, (dw, dh) = letterbox(img, new_shape=(IMG_SIZE, IMG_SIZE)) img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) img = cv2.resize(img, (IMG_SIZE, IMG_SIZE)) # Inference outputs = rknn.inference(inputs=[img]) # post process input0_data = outputs[0] input1_data = outputs[1] input2_data = outputs[2] input0_data = input0_data.reshape([3, -1]+list(input0_data.shape[-2:])) input1_data = input1_data.reshape([3, -1]+list(input1_data.shape[-2:])) input2_data = input2_data.reshape([3, -1]+list(input2_data.shape[-2:])) input_data = list() input_data.append(np.transpose(input0_data, (2, 3, 0, 1))) input_data.append(np.transpose(input1_data, (2, 3, 0, 1))) input_data.append(np.transpose(input2_data, (2, 3, 0, 1))) boxes, classes, scores = yolov5_post_process(input_data) img_1 = cv2.cvtColor(img, cv2.COLOR_RGB2BGR) if boxes is not None: draw(img_1, boxes, scores, classes) # show output cv2.imshow("post process result", img_1) cv2.waitKey(0) cv2.destroyAllWindows() rknn.release()这是基于原来test.py做的一些修改，删除了导出模型的部分，只保留了加载模型和推理部分。接下来把这几个文件放在同一个文件夹，传到香橙派的 /home/orangepi/Desktop/yolov5 目录下
https://github.com/rockchip-linux/rknn-toolkit2/tree/v1.5.2
第二步——在香橙派上部署
（1）将rknn-toolkit2-1.5.2.zip上传到香橙派上并解压，上传到Download目录下面
在香橙派上创建python=3.10.0的虚拟环境（yolo_py310），进入虚拟环境
将①版本升级为1.5.2，结合下面这个两个网站进行升级
rknn3588如何升级驱动_rk2588 rknpu驱动-CSDN博客
RK3588S rknnlite版本1.4升级到1.5.2，yolov5推理速度变慢 · Issue #242 · rockchip-linux/rknn-toolkit2 (github.com)
将②版本升级到0.9.6，这个博客的做法可以升级，升级后要重启
（此方法存疑，我是按照这个博客来的，但有人失败了，可以去github上面寻求答案）
香橙派5B-RK3588-NPU内核更新至0.9.6_rk3588更新npu驱动-CSDN博客
（2）下载rknn-toolkit2-1.5.2依赖
将下载源设置为清华源
pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simplepip install rknn/rknn-toolkit2-lite/packages/rknn_toolkit_lite2-1.5.2-cp310-cp310-linux_aarch64.whl安装剩下的依赖包，numpy的版本要1.x的
pip install opencv-python=4.5.5.64运行deploy.py
python deploy.py运行之后看到这个图像就是部署成功了
1万+
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
