高通QCS6490平台部署YOLOv8,Opencv C++读取视频/USB摄像头推理识别测试

高通QCS6490平台部署YOLOv8,Opencv C++读取视频/USB摄像头推理识别测试

1. 参考说明

高通QCS6490开发板： APLUX犀牛派AI开发板

平台手册：犀牛派A1 产品使用手册 | APLUX Doc Center；

平台SDK：AidLite SDK | APLUX Doc Center

平台模型广场：模型广场

高通QNN文档：AI Engine Direct SDK

2. 操作说明

2.1 供电与连接

板卡为 DC 12V供电，使用网络访问板卡：

2.2 网络访问

通过adb工具将板卡IP修改为：192.168.1.xxx，

可通过ssh访问：

ip: 192.168.1.xxx

账号：aidlux

密码：aidlux

通过浏览器访问：

url: http://192.168.1.xxx:8000

账号：aidlux

密码：aidlux

验证AidLite安装(Linux)

# aidlite sdk c++ check
python3 -c "import aidlite; print(aidlite.get_library_version())"

# aidlite sdk python check
python3 -c "import aidlite; print(aidlite.get_py_library_version())"

3. 算法验证

内置ubuntu22.04系统，默认安装了python3.10、opencv等库，可进行yolov5、yolov8、yolov11等官方demo验证，demo来自官方模型广场：模型广场，选择 QCS6490芯片平台，根据需求选择算法，然后将下载的demo拷贝到板卡对应路径下，进行编译和测试：

3.1 模型转换

pt转换为onnx模型

# conda 新建环境
conda create -n Model_T python=3.10
# 打开环境
conda activate Model_T
# 配置包
pip install ultralytics onnx

新建一个python文件，用于模型转换以及导出：

## model.py
from ultralytics import YOLO

model = YOLO('./yolov8n.pt') # 加载pt模型文件
# 导出ONNX配置参数
export_params = {
   
   
'format': 'onnx',
'opset': 12, 		# 推荐算子集版本
'simplify': True, 	# 启用模型简化
'dynamic': False, 	# 固定输入尺寸
'imgsz': 640, 		# 标准输入尺寸
'half': False 		# 保持FP32精度
} 
#执行转换并保存到同级目录
model.export(**export_params)

Model_T 环境下,执行命令,转换pt文件为onnx

python model.py

3.2 模型优化

参考 模型优化平台 (AIMO) 用户指南 和模型广场yolov8的“模型转换参考”。
使用 模型优化平台 (AIMO) 进行模型转换，转换完成后下载到本地，然后在拷贝到板卡yolov8测试源码models加载路径下。

yolov8 模型测试

# c++
cd /home/aidlux/aidlite_demo/examples/model_farm_yolov8s_qcs6490_qnn2.31_int8_aidlite/cpp
mkdir build
cd build/
cmake ..
make -j
./run_test --target_model ../../models/cutoff_yolov8n_qcs6490_w8a8.qnn231.ctx.bin --imgs ../bus.jpg  --invoke_nums 10

#python
cd /home/aidlux/aidlite_demo/examples/model_farm_yolov8s_qcs6490_qnn2.31_int8_aidlite/
python3  python/run_test.py --target_model ./models/cutoff_yolov8s_qcs6490_w8a8.qnn231.ctx.bin --imgs ./python/bus.jpg  --invoke_nums 10

3.3 修改c++源码为读取mp4视频或USB摄像头验证

查看usb摄像头设备

执行 ls /dev/video* 后看到以下设备：

我这里USB摄像头设备默认不是0,可通过v4l2-ctl 查看每个设备的信息

v4l2-ctl --device=/dev/video0 --info
v4l2-ctl --device=/dev/video1 --info
v4l2-ctl --device=/dev/video2 --info
v4l2-ctl --device=/dev/video3 --info

发现usb摄像头设备ID是 2

c++源码如下:

#include <iostream>
#include <string>
#include <algorithm>
#include <cctype>
#include <opencv2/opencv.hpp>
#include <aidlux/aidlite/aidlite.hpp>
#include <vector>
#include <numeric>
#include <signal.h>
#include <chrono>


const float INPUT_WIDTH = 640.0;
const float INPUT_HEIGHT = 640.0;
const float SCORE_THRESHOLD = 0.25;
const float NMS_THRESHOLD = 0.45;
const float CONFIDENCE_THRESHOLD = 0.25;
const uint32_t size = 640;
const uint32_t out_size = 8400;

const int FONT_FACE = cv::FONT_HERSHEY_SIMPLEX;
cv::Scalar WHITE = cv::Scalar(255,255,255);

const float FONT_SCALE = 0.75;
const int THICKNESS = 1;

const int CLASS_NUM = 80;  //目标类数量

const std::vector<std::string> class_list = {
   
   
    "person", "bicycle", "car", "motorcycle", "airplane", "bus", "train",
    "truck", "boat", "traffic light", "fire hydrant", "stop sign", "parking meter",
    "bench", "bird", "cat", "dog", "horse", "sheep", "cow", "elephant", "bear",
    "zebra", "giraffe", "backpack", "umbrella", "handbag", "tie", "suitcase",
    "frisbee", "skis", "snowboard", "sports ball", "kite", "baseball bat",
    "baseball glove", "skateboard", "surfboard", "tennis racket", "bottle",
    "wine glass", "cup", "fork", "knife", "spoon", "bowl", "banana", "apple",
    "sandwich", "orange", "broccoli", "carrot", "hot dog", "pizza", "donut",
    "cake", "chair", "couch", "potted plant", "bed", "dining table", "toilet",
    "TV", "laptop", "mouse", "remote", "keyboard", "cell phone", "microwave",
    "oven", "toaster", "sink", "refrigerator", "book", "clock", "vase",
    "scissors", "teddy bear", "hair drier", "toothbrush"
};


// const std::vector<std::string> class_list = {
   
   
//     "Multi-rotor-UAV","Fixed-wing-UAV"
// };


using namespace Aidlux::Aidlite;


// 输出视频路径   
std::string output_video = "./output.mp4"