超级会员免费看
RK3588 Android13 运行yolov5 进行物体识别
项目上要用到物体识别，打算用yolov5来做
网上使用 RK3588 搭建yolov5的博客有很多，但是大部分都是运行LinuxSDK上。我需要搭在Android13上使用，目前先确保有一个比较简单的demo能运行起来
YOLOv5简介：
YOLOv5是一种基于深度学习的目标检测算法，其核心思想是将目标检测视为回归问题，通过卷积神经网络直接预测目标的位置和类别。相比传统的目标检测算法，YOLOv5 具有更高的检测速度和准确性，尤其适用于实时目标检测任务。它的主要特点包括：
- Anchor-free 设计：传统目标检测算法需要先确定物体位置并给出候选框，但 YOLOv5 采用了无锚设计方式，直接预测物体的位置和大小，避免了候选框对检测性能的影响。
- 多尺度检测：YOLOv5 可以精确检测不同尺度、各种形状和姿态的目标，具有很好的适应性。
- 目标定位精确：通过导出中心点坐标来实现目标的精准定位，并在分类和回归两个方面进行优化，提高了目标检测精度。
- 快速检测速度：采用高效计算方法和 GPU 等硬件加速技术，保证高精度的同时具有非常快的检测速度。
SOC：RK3588
system：Android13
一.下载yolov5
1.先下载yolov5
git clone https://github.com/ultralytics/yolov5.git2.安装yolov5的python依赖
cd yolov5/
pip install -r requirements.txt
时间可能会有点长
3.训练官方提供的数据集
python3 train.py --img 640 --batch 16 --epochs 3 --data coco128.yaml --weights yolov5s.pt结束：
microwave 128 3 0.826 1 0.995 0.703
oven 128 5 0.462 0.4 0.441 0.304
sink 128 6 0.356 0.167 0.364 0.287
refrigerator 128 5 0.664 0.8 0.808 0.548
book 128 29 0.659 0.2 0.317 0.152
clock 128 9 0.727 0.778 0.889 0.757
vase 128 2 0.484 1 0.995 0.945
scissors 128 1 1 0 0.995 0.199
teddy bear 128 21 0.787 0.524 0.767 0.512
toothbrush 128 5 0.815 0.6 0.928 0.59
Results saved to runs/train/exp9
*@*-desktop:~/workSpace/rk3588_yolo/yolov5$拷贝自己训练的文件到当前目录
*@*-desktop:~/workSpace/rk3588_yolo/yolov5$ cp runs/train/exp9/weights/best.pt ./4.导出转换onnx模型文件
打下面patch
--- a/models/yolo.py
+++ b/models/yolo.py
@@ -88,31 +88,38 @@ class Detect(nn.Module):
self.m = nn.ModuleList(nn.Conv2d(x, self.no * self.na, 1) for x in ch) # output conv
self.inplace = inplace # use inplace ops (e.g. slice assignment)
+ # def forward(self, x):
+ # """Processes input through YOLOv5 layers, altering shape for detection: `x(bs, 3, ny, nx, 85)`."""
+ # z = [] # inference output
+ # for i in range(self.nl):
+ # x[i] = self.m[i](x[i]) # conv
+ # bs, _, ny, nx = x[i].shape # x(bs,255,20,20) to x(bs,3,20,20,85)
+ # x[i] = x[i].view(bs, self.na, self.no, ny, nx).permute(0, 1, 3, 4, 2).contiguous()
+
+ # if not self.training: # inference
+ # if self.dynamic or self.grid[i].shape[2:4] != x[i].shape[2:4]:
+ # self.grid[i], self.anchor_grid[i] = self._make_grid(nx, ny, i)
+
+ # if isinstance(self, Segment): # (boxes + masks)
+ # xy, wh, conf, mask = x[i].split((2, 2, self.nc + 1, self.no - self.nc - 5), 4)
+ # xy = (xy.sigmoid() * 2 + self.grid[i]) * self.stride[i] # xy
+ # wh = (wh.sigmoid() * 2) ** 2 * self.anchor_grid[i] # wh
+ # y = torch.cat((xy, wh, conf.sigmoid(), mask), 4)
+ # else: # Detect (boxes only)
+ # xy, wh, conf = x[i].sigmoid().split((2, 2, self.nc + 1), 4)
+ # xy = (xy * 2 + self.grid[i]) * self.stride[i] # xy
+ # wh = (wh * 2) ** 2 * self.anchor_grid[i] # wh
+ # y = torch.cat((xy, wh, conf), 4)
+ # z.append(y.view(bs, self.na * nx * ny, self.no))
+
+ # return x if self.training else (torch.cat(z, 1),) if self.export else (torch.cat(z, 1), x)
def forward(self, x):
- """Processes input through YOLOv5 layers, altering shape for detection: `x(bs, 3, ny, nx, 85)`."""
z = [] # inference output
for i in range(self.nl):
- x[i] = self.m[i](x[i]) # conv
- bs, _, ny, nx = x[i].shape # x(bs,255,20,20) to x(bs,3,20,20,85)
- x[i] = x[i].view(bs, self.na, self.no, ny, nx).permute(0, 1, 3, 4, 2).contiguous()
-
- if not self.training: # inference
- if self.dynamic or self.grid[i].shape[2:4] != x[i].shape[2:4]:
- self.grid[i], self.anchor_grid[i] = self._make_grid(nx, ny, i)
-
- if isinstance(self, Segment): # (boxes + masks)
- xy, wh, conf, mask = x[i].split((2, 2, self.nc + 1, self.no - self.nc - 5), 4)
- xy = (xy.sigmoid() * 2 + self.grid[i]) * self.stride[i] # xy
- wh = (wh.sigmoid() * 2) ** 2 * self.anchor_grid[i] # wh
- y = torch.cat((xy, wh, conf.sigmoid(), mask), 4)
- else: # Detect (boxes only)
- xy, wh, conf = x[i].sigmoid().split((2, 2, self.nc + 1), 4)
- xy = (xy * 2 + self.grid[i]) * self.stride[i] # xy
- wh = (wh * 2) ** 2 * self.anchor_grid[i] # wh
- y = torch.cat((xy, wh, conf), 4)
- z.append(y.view(bs, self.na * nx * ny, self.no))
-
- return x if self.training else (torch.cat(z, 1),) if self.export else (torch.cat(z, 1), x)
+ if os.getenv('RKNN_model_hack', '0') != '0':
+ x[i] = torch.sigmoid(self.m[i](x[i])) # conv
+
+ return x
def _make_grid(self, nx=20, ny=20, i=0, torch_1_10=check_version(torch.__version__, "1.10.0")):
"""Generates a mesh grid for anchor boxes with optional compatibility for torch versions < 1.10."""--- a/export.py
+++ b/export.py
@@ -214,10 +215,10 @@ def try_export(inner_func):
import pandas as pd
import torch
from torch.utils.mobile_optimizer import optimize_for_mobile
+os.environ['RKNN_model_hack'] = 'npu_2'
@@ -1400,7 +1401,8 @@ def run(
- shape = tuple((y[0] if isinstance(y, tuple) else y).shape) # model output shape
+ # shape = tuple((y[0] if isinstance(y, tuple) else y).shape) # model output shape
+ shape = tuple(y[0].shape) python3 export.py --weights best.pt --data data/coco128.yaml --include onnx --opset 12
到这里yolov5已经结束，接下来需要下载RK的rknn-toolkit2 工具将训练好的yolov5 onnx模型转换为rknn模型
二.下载rknn-toolkit2
git clone https://github.com/rockchip-linux/rknn-toolkit2.git
由于RK3588 rknn-toolkit2 使用的python环境与yolov5的 python环境不同，这时需要安装coda（能支持多个python环境），安装可以参考这篇博客超详细Ubuntu安装Anaconda步骤+Anconda常用命令_ubuntu 安装anaconda-CSDN博客
1.进入coda并 安装rknn-toolkit2的环境：
conda activate base
pip install ./rknn-toolkit2/packages/rknn_toolkit2-2.0.0b0+9bab5682-cp311-cp311-linux_x86_64.whl //这里要确认自己python版本 我是3.11 运行cp311
2.修改
--- a/rknn-toolkit2/examples/onnx/yolov5/test.py
+++ b/rknn-toolkit2/examples/onnx/yolov5/test.py
@@ -8,8 +8,8 @@ import cv2
from rknn.api import RKNN
# Model from https://github.com/airockchip/rknn_model_zoo
-ONNX_MODEL = 'yolov5s_relu.onnx'
-RKNN_MODEL = 'yolov5s_relu.rknn'
+ONNX_MODEL = 'best.onnx'
+RKNN_MODEL = 'best.rknn'
IMG_PATH = './bus.jpg'
DATASET = './dataset.txt'
@@ -239,7 +239,7 @@ if __name__ == '__main__':
# pre-process config
print('--> Config model')
- rknn.config(mean_values=[[0, 0, 0]], std_values=[[255, 255, 255]], target_platform='rk3566')
+ rknn.config(mean_values=[[0, 0, 0]], std_values=[[255, 255, 255]], target_platform='rk3588')
print('done')
# Load ONNX model将上次生成的onnx 模型拷贝到当前 并运行test.py ,生成best.rknn
examples/onnx/yolov5$ cd rknn-toolkit2/examples/onnx/yolov5/
examples/onnx/yolov5$ cp ~/workSpace/rk3588_yolo/yolov5/best.onnx ./
examples/onnx/yolov5$ python test.py
....
done
class score xmin, ymin, xmax, ymax
--------------------------------------------------
person 0.840 [ 113, 231, 206, 550]
person 0.761 [ 211, 243, 283, 521]
person 0.714 [ 473, 228, 561, 525]
person 0.362 [ 77, 334, 121, 515]
parking meter 0.210 [ 470, 371, 488, 419]
bus 0.798 [ 87, 134, 549, 483]
Save results to result.jpg!
examples/onnx/yolov5$ ls
best.onnx bus.jpg check3_fuse_ops.onnx model_config.yml onnx_yolov5_1.npy README.md result_truth.jpg yolov5s_relu.onnx
best.rknn check0_base_optimize.onnx dataset.txt onnx_yolov5_0.npy onnx_yolov5_2.npy result.jpg test.py yolov5s_rk3588.rknn三.拷贝到RK的RKNN Android demo上
1.下载rknpu2
git clone https://github.com/rockchip-linux/rknpu2.git2.将我们自己生成的rknn 替换至demo
cp best.rknn rknpu2/examples/rknn_yolov5_android_apk_demo/app/src/main/res/raw/yolov5s_rk3588.rknn3.打开Android Studio 运行demo
加点打印
--- a/examples/rknn_yolov5_android_apk_demo/app/src/main/java/com/rockchip/gpadc/demo/CameraPreviewActivity.java
+++ b/examples/rknn_yolov5_android_apk_demo/app/src/main/java/com/rockchip/gpadc/demo/CameraPreviewActivity.java
@@ -546,7 +546,7 @@ public class CameraPreviewActivity extends Activity implements Camera.PreviewCal
detection.top *= height;
detection.bottom *= height;
-// Log.d(TAG, rego.toString());
+ Log.d(TAG, " :"+rego.getTrackId() + " - " + mInferenceResult.mPostProcess.getLabelTitle(rego.getId()));
+ Log.d(TAG, detection.toString());
//
3614
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
