环境:
yolo v8版本:8.0.151
1.训练自己的模型(省)
from ultralytics import YOLO
if __name__ == '__main__':
# 加载模型
model = YOLO(r'yolov8-seg.yaml') # 不使用预训练权重训练
# model = YOLO(r'yolov8-seg.yaml').load("yolov8n-seg.pt") # 使用预训练权重训练
# 训练参数 ----------------------------------------------------------------------------------------------
model.train(
data=r'coco128bak-seg.yaml',
epochs=300, # (int) 训练的周期数
batch=64, # (int) 每批次的图像数量(-1 为自动批处理)
device='', # (int | str | list, optional) 运行的设备,例如 cuda device=0 或 device=0,1,2,3 或 device=cpu
workers=8, # (int) 数据加载的工作线程数(每个DDP进程)
)
2.pt模型转onnx模型再转rknn模型
a.修改 ultralytics/cfg/defualt.yaml中的model字段
model: 你训练后生成的模型文件路径/best.pt
b.执行ultralytics/engine/exporter.py将.pt模型转为onnx模型,生成的模型会在打印信息中给出。
python ultralytics/engine/exporter.py
c.借助toolkit2将onnx转为rknn模型,此处省略toolkit2的相关安装,参见:[RK git](https://github.com/rockchip-linux/rknn-toolkit2)
转换命令例如:python convert.py yolov8-seg.onnx rk3588
3.交叉编译C++部署代码
rk已经提供了参考的部署代码:rknn_model_zoo仓库
clone代码下来,在rknn_model_zoo/build-linux.sh中修改交叉编译工具链路径就可以编译官方deme了
例如本人修改下:
执行:
chmod 777 build-linux.sh
./build-linux.sh -t rk3588 -a aarch64 -d yolov8
编译的文件会全部打包到install,修改install/rk3588_linux_aarch64/rknn_yolov8_seg_demo/model/coco_80_labels_list.txt中的内容,修改为自己的类别,将上述所转换的rknn模型与该文件压缩打包到rk3588平台上就可以了
4.部署到rk3588平台
在平台上根据程序执行的参数格式运行程序
./rknn_yolov8_seg_demo model/yolov8-seg.rknn test.jpg
执行效果:
load lable ./model/coco_80_labels_list.txt
model input num: 1, output num: 13
input tensors:
index=0, name=images, n_dims=4, dims=[1, 640, 640, 3], n_elems=1228800, size=1228800, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
output tensors:
index=0, name=375, n_dims=4, dims=[1, 64, 80, 80], n_elems=409600, size=409600, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-3, scale=0.113323
index=1, name=onnx::ReduceSum_383, n_dims=4, dims=[1, 10, 80, 80], n_elems=64000, size=64000, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003346
index=2, name=388, n_dims=4, dims=[1, 1, 80, 80], n_elems=6400, size=6400, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003505
index=3, name=354, n_dims=4, dims=[1, 32, 80, 80], n_elems=204800, size=204800, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-9, scale=0.023833
index=4, name=395, n_dims=4, dims=[1, 64, 40, 40], n_elems=102400, size=102400, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=7, scale=0.094574
index=5, name=onnx::ReduceSum_403, n_dims=4, dims=[1, 10, 40, 40], n_elems=16000, size=16000, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003012
index=6, name=407, n_dims=4, dims=[1, 1, 40, 40], n_elems=1600, size=1600, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003036
index=7, name=361, n_dims=4, dims=[1, 32, 40, 40], n_elems=51200, size=51200, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-5, scale=0.022464
index=8, name=414, n_dims=4, dims=[1, 64, 20, 20], n_elems=25600, size=25600, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=3, scale=0.068694
index=9, name=onnx::ReduceSum_422, n_dims=4, dims=[1, 10, 20, 20], n_elems=4000, size=4000, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.002477
index=10, name=426, n_dims=4, dims=[1, 1, 20, 20], n_elems=400, size=400, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.002487
index=11, name=368, n_dims=4, dims=[1, 32, 20, 20], n_elems=12800, size=12800, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-3, scale=0.018722
index=12, name=347, n_dims=4, dims=[1, 32, 160, 160], n_elems=819200, size=819200, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-117, scale=0.025950
model is NHWC input fmt
model input height=640, width=640, channel=3
origin size=3984x1840 crop size=3984x1840
input image: 3984 x 1840, subsampling: 4:2:0, colorspace: YCbCr, orientation: 1
scale=0.160643 dst_box=(0 172 639 465) allow_slight_change=1 _left_offset=0 _top_offset=172 padding_w=0 padding_h=346
src width=3984 height=1840 fmt=0x1 virAddr=0x0x7fbbf9e010 fd=0
dst width=640 height=640 fmt=0x1 virAddr=0x0x38ec4000 fd=0
src_box=(0 0 3983 1839)
dst_box=(0 172 639 465)
color=0x72
rga_api version 1.10.0_[2]
fill dst image (x y w h)=(0 0 640 640) with color=0x72727272
RgaCollorFill(1756) RGA_COLORFILL fail: Invalid argument
RgaCollorFill(1757) RGA_COLORFILL fail: Invalid argument
230 im2d_rga_impl rga_task_submit(2100): Failed to call RockChipRga interface, please use 'dmesg' command to view driver error log.
230 im2d_rga_impl rga_dump_channel_info(1452): src_channel:
rect[x,y,w,h] = [0, 0, 0, 0]
image[w,h,ws,hs,f] = [0, 0, 0, 0, rgba8888]
buffer[handle,fd,va,pa] = [0, 0, 0, 0]
color_space = 0x0, global_alpha = 0x0, rd_mode = 0x0
230 im2d_rga_impl rga_dump_channel_info(1452): dst_channel:
rect[x,y,w,h] = [0, 0, 640, 640]
image[w,h,ws,hs,f] = [640, 640, 640, 640, rgb888]
buffer[handle,fd,va,pa] = [108, 0, 0, 0]
color_space = 0x0, global_alpha = 0xff, rd_mode = 0x1
230 im2d_rga_impl rga_dump_opt(1502): opt version[0x0]:
230 im2d_rga_impl rga_dump_opt(1503): set_core[0x0], priority[0]
230 im2d_rga_impl rga_dump_opt(1506): color[0x72727272]
230 im2d_rga_impl rga_dump_opt(1515):
230 im2d_rga_impl rga_task_submit(2109): acquir_fence[-1], release_fence_ptr[0x0], usage[0x280000]
rknn_run
carpet @ (2975 902 3965 1475) 0.768
wire @ (1170 547 2203 1425) 0.768
null @ (946 31 1145 168) 0.512
write_image path: out.png width=3984 height=1840 channel=3 data=0x7fbbf9e010
参考:
https://doc.embedfire.com/linux/rk356x/Ai/zh/latest/README.html
参考的yolo v8代码:https://gitee.com/ysxgitee/rk3588_npu_yolov8_2.git