Qualcomm Robotics RB5 平台软件用户指南（12）

本文链接：https://blog.csdn.net/weixin_38498942/article/details/138191487

博客介绍了图像分类用例，包括单摄像头流和单文件流，且都带有AI推理。详细阐述了使用TensorFlow Lite和Qualcomm神经处理SDK执行用例的步骤，涉及不同显示和编码变体在不同来源及系统下的情况，还说明了验证方法，如保存文件和检查视频流。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

Qualcomm Robotics RB5 平台软件用户指南（12）

- 5.9 图像分类用例

5.9 图像分类用例

用例
- 单摄像头流：1080p，带 AI 推理。
- 单文件流：720p，带 AI 推理。

TensorFlow Lite：执行用例的步骤

显示（变体 1）：使用qtioverlay插件应用分类叠加。

相机来源：

Linux Ubuntu：

export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtimetamux name=metamux ! queue ! qtioverlay ! queue ! waylandsink sync=false fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/mobilenet_v2_1.0_224_quant.tflite ! queue ! qtimlvclassification threshold=60.0 results=3 module=mobilenet labels=/data/mobilenet.labels ! text/x-raw ! queue ! metamux.

嵌入式Linux：

export WAYLAND_DISPLAY=wayland-1 && export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtimetamux name=metamux ! queue ! qtioverlay ! queue ! waylandsink sync=false fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/mobilenet_v2_1.0_224_quant.tflite ! queue ! qtimlvclassification threshold=60.0 results=3 module=mobilenet labels=/data/mobilenet.labels ! text/x-raw ! queue ! metamux.

文件来源：

Linux Ubuntu：

export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/Animals_003_720p_180s_30FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtimetamux name=metamux ! queue ! qtioverlay ! queue ! waylandsink sync=true fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/mobilenet_v2_1.0_224_quant.tflite ! queue ! qtimlvclassification threshold=60.0 results=3 module=mobilenet labels=/data/mobilenet.labels ! text/x-raw ! queue ! metamux.

嵌入式Linux：

export WAYLAND_DISPLAY=wayland-1 && export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/Animals_003_720p_180s_30FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtimetamux name=metamux ! queue ! qtioverlay ! queue ! waylandsink sync=true fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/mobilenet_v2_1.0_224_quant.tflite ! queue ! qtimlvclassification threshold=60.0 results=3 module=mobilenet labels=/data/mobilenet.labels ! text/x-raw ! queue ! metamux.

显示（变体 2）：使用qtivcomposer插件将原始帧与分类掩码混合。

相机来源：

Linux Ubuntu：

export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::position="<50, 50>" sink_1::dimensions="<368, 64>" ! queue ! waylandsink sync=false fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/mobilenet_v2_1.0_224_quant.tflite ! queue ! qtimlvclassification threshold=60.0 results=3 module=mobilenet labels=/data/mobilenet.labels ! video/x-raw,format=BGRA,width=368,height=64 ! queue ! mixer.

嵌入式Linux：

export WAYLAND_DISPLAY=wayland-1 && export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::position="<50, 50>" sink_1::dimensions="<368, 64>" ! queue ! waylandsink sync=false fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/mobilenet_v2_1.0_224_quant.tflite ! queue ! qtimlvclassification threshold=60.0 results=3 module=mobilenet labels=/data/mobilenet.labels ! video/x-raw,format=BGRA,width=368,height=64 ! queue ! mixer.

文件来源：

Linux Ubuntu：

export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/Animals_003_720p_180s_30FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::position="<50, 50>" sink_1::dimensions="<368, 64>" ! queue ! waylandsink sync=true fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/mobilenet_v2_1.0_224_quant.tflite ! queue ! qtimlvclassification threshold=60.0 results=3 module=mobilenet labels=/data/mobilenet.labels ! video/x-raw,format=BGRA,width=368,height=64 ! queue ! mixer.

嵌入式Linux：

export WAYLAND_DISPLAY=wayland-1 && export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/Animals_003_720p_180s_30FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::position="<50, 50>" sink_1::dimensions="<368, 64>" ! queue ! waylandsink sync=true fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/mobilenet_v2_1.0_224_quant.tflite ! queue ! qtimlvclassification threshold=60.0 results=3 module=mobilenet labels=/data/mobilenet.labels ! video/x-raw,format=BGRA,width=368,height=64 ! queue ! mixer.

编码（变体 1）：使用qtioverlay插件应用分类叠加。

相机来源：

gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtimetamux name=metamux ! queue ! qtioverlay ! queue ! qtic2venc target-bitrate=6000000 ! h264parse ! queue ! mp4mux ! queue ! filesink location=/data/video.mp4 split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/mobilenet_v2_1.0_224_quant.tflite ! queue ! qtimlvclassification threshold=60.0 results=3 module=mobilenet labels=/data/mobilenet.labels ! text/x-raw ! queue ! metamux.

文件来源：

gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/Animals_003_720p_180s_30FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtimetamux name=metamux ! queue ! qtioverlay ! queue ! qtic2venc target-bitrate=6000000 ! h264parse ! queue ! mp4mux ! queue ! filesink location=/data/video.mp4 split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/mobilenet_v2_1.0_224_quant.tflite ! queue ! qtimlvclassification threshold=60.0 results=3 module=mobilenet labels=/data/mobilenet.labels ! text/x-raw ! queue ! metamux.

编码（变体 2）：使用qtivcomposer插件将原始帧与分类掩码混合。

相机来源：

gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::position="<50, 50>" sink_1::dimensions="<368, 64>" ! queue ! qtic2venc target-bitrate=6000000 ! h264parse ! queue ! mp4mux ! queue ! filesink location=/data/video.mp4 split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/mobilenet_v2_1.0_224_quant.tflite ! queue ! qtimlvclassification threshold=60.0 results=3 module=mobilenet labels=/data/mobilenet.labels ! video/x-raw,format=BGRA,width=368,height=64 ! queue ! mixer.

文件来源：

gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/Animals_003_720p_180s_30FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::position="<50, 50>" sink_1::dimensions="<368, 64>" ! queue ! qtic2venc target-bitrate=6000000 ! h264parse ! queue ! mp4mux ! queue ! filesink location=/data/video.mp4 split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/mobilenet_v2_1.0_224_quant.tflite ! queue ! qtimlvclassification threshold=60.0 results=3 module=mobilenet labels=/data/mobilenet.labels ! video/x-raw,format=BGRA,width=368,height=64 ! queue ! mixer.

Qualcomm 神经处理 SDK：执行用例的步骤

显示（变体 1）：使用qtioverlay插件应用分类叠加。

相机来源：

Linux Ubuntu：

export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtimetamux name=metamux ! queue ! qtioverlay text-font-size=24 ! queue ! waylandsink sync=false fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/mobilenet_v1_quantaware_quantized.dlc ! queue ! qtimlvclassification threshold=51.0 results=1 module=mobilenet labels=/data/mobilenet.labels ! text/x-raw ! queue ! metamux.

嵌入式Linux：

export WAYLAND_DISPLAY=wayland-1 && export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtimetamux name=metamux ! queue ! qtioverlay text-font-size=24 ! queue ! waylandsink sync=false fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/mobilenet_v1_quantaware_quantized.dlc ! queue ! qtimlvclassification threshold=51.0 results=1 module=mobilenet labels=/data/mobilenet.labels ! text/x-raw ! queue ! metamux.

文件来源：

Linux Ubuntu：

export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/Animals_003_720p_180s_30FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtimetamux name=metamux ! queue ! qtioverlay text-font-size=24 ! queue ! waylandsink sync=true fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/mobilenet_v1_quantaware_quantized.dlc ! queue ! qtimlvclassification threshold=51.0 results=1 module=mobilenet labels=/data/mobilenet.labels ! text/x-raw ! queue ! metamux.

嵌入式Linux：

export WAYLAND_DISPLAY=wayland-1 && export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/Animals_003_720p_180s_30FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtimetamux name=metamux ! queue ! qtioverlay text-font-size=24 ! queue ! waylandsink sync=true fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/mobilenet_v1_quantaware_quantized.dlc ! queue ! qtimlvclassification threshold=51.0 results=1 module=mobilenet labels=/data/mobilenet.labels ! text/x-raw ! queue ! metamux.

显示（变体 2）：使用qtivcomposer插件将原始帧与分类掩码混合。

相机来源：

Linux Ubuntu：

export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::position="<50, 50>" sink_1::dimensions="<368, 32>" ! queue ! waylandsink sync=false fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/mobilenet_v1_quantaware_quantized.dlc ! queue ! qtimlvclassification threshold=51.0 results=1 module=mobilenet labels=/data/mobilenet.labels ! video/x-raw,format=BGRA,width=368,height=32 ! queue ! mixer.

嵌入式Linux：

export WAYLAND_DISPLAY=wayland-1 && export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::position="<50, 50>" sink_1::dimensions="<368, 32>" ! queue ! waylandsink sync=false fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/mobilenet_v1_quantaware_quantized.dlc ! queue ! qtimlvclassification threshold=51.0 results=1 module=mobilenet labels=/data/mobilenet.labels ! video/x-raw,format=BGRA,width=368,height=32 ! queue ! mixer.

文件来源：

Linux Ubuntu：

export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/Animals_003_720p_180s_30FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::position="<50, 50>" sink_1::dimensions="<368, 32>" ! queue ! waylandsink sync=true fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/mobilenet_v1_quantaware_quantized.dlc ! queue ! qtimlvclassification threshold=51.0 results=1 module=mobilenet labels=/data/mobilenet.labels ! video/x-raw,format=BGRA,width=368,height=32 ! queue ! mixer.

嵌入式Linux：

export WAYLAND_DISPLAY=wayland-1 && export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/Animals_003_720p_180s_30FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::position="<50, 50>" sink_1::dimensions="<368, 32>" ! queue ! waylandsink sync=true fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/mobilenet_v1_quantaware_quantized.dlc ! queue ! qtimlvclassification threshold=51.0 results=1 module=mobilenet labels=/data/mobilenet.labels ! video/x-raw,format=BGRA,width=368,height=32 ! queue ! mixer.

编码（变体 1）：使用qtioverlay插件应用分类叠加。

相机来源：

gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtimetamux name=metamux ! queue ! qtioverlay text-font-size=24 ! queue ! qtic2venc target-bitrate=6000000 ! h264parse ! queue ! mp4mux ! queue ! filesink location=/data/video.mp4 split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/mobilenet_v1_quantaware_quantized.dlc ! queue ! qtimlvclassification threshold=51.0 results=1 module=mobilenet labels=/data/mobilenet.labels ! text/x-raw ! queue ! metamux.

文件来源：

gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/Animals_003_720p_180s_30FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtimetamux name=metamux ! queue ! qtioverlay text-font-size=24 ! queue ! qtic2venc target-bitrate=6000000 ! h264parse ! queue ! mp4mux ! queue ! filesink location=/data/video.mp4 split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/mobilenet_v1_quantaware_quantized.dlc ! queue ! qtimlvclassification threshold=51.0 results=1 module=mobilenet labels=/data/mobilenet.labels ! text/x-raw ! queue ! metamux.

编码（变体 2）：使用qtivcomposer插件将原始帧与分类掩码混合。

相机来源：

gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::position="<50, 50>" sink_1::dimensions="<368, 32>" ! queue ! qtic2venc target-bitrate=6000000 ! h264parse ! queue ! mp4mux ! queue ! filesink location=/data/video.mp4 split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/mobilenet_v1_quantaware_quantized.dlc ! queue ! qtimlvclassification threshold=51.0 results=1 module=mobilenet labels=/data/mobilenet.labels ! video/x-raw,format=BGRA,width=368,height=32 ! queue ! mixer.

文件来源：

gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/Animals_003_720p_180s_30FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::position="<50, 50>" sink_1::dimensions="<368, 32>" ! queue ! qtic2venc target-bitrate=6000000 ! h264parse ! queue ! mp4mux ! queue ! filesink location=/data/video.mp4 split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/mobilenet_v1_quantaware_quantized.dlc ! queue ! qtimlvclassification threshold=51.0 results=1 module=mobilenet labels=/data/mobilenet.labels ! video/x-raw,format=BGRA,width=368,height=32 ! queue ! mixer.