Qualcomm Robotics RB5 平台软件用户指南(12)

150 篇文章 13 订阅
25 篇文章 1 订阅

Qualcomm Robotics RB5 平台软件用户指南(12)


5.9 图像分类用例

  • 用例

    • 单摄像头流:1080p,带 AI 推理。
    • 单文件流:720p,带 AI 推理。
  • TensorFlow Lite:执行用例的步骤

    • 显示(变体 1):使用qtioverlay插件应用分类叠加。
      • 相机来源:
        • Linux Ubuntu:
          export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtimetamux name=metamux ! queue ! qtioverlay ! queue ! waylandsink sync=false fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/mobilenet_v2_1.0_224_quant.tflite ! queue ! qtimlvclassification threshold=60.0 results=3 module=mobilenet labels=/data/mobilenet.labels ! text/x-raw ! queue ! metamux.
          
        • 嵌入式Linux:
          export WAYLAND_DISPLAY=wayland-1 && export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtimetamux name=metamux ! queue ! qtioverlay ! queue ! waylandsink sync=false fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/mobilenet_v2_1.0_224_quant.tflite ! queue ! qtimlvclassification threshold=60.0 results=3 module=mobilenet labels=/data/mobilenet.labels ! text/x-raw ! queue ! metamux.
          
      • 文件来源:
        • Linux Ubuntu:
          export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/Animals_003_720p_180s_30FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtimetamux name=metamux ! queue ! qtioverlay ! queue ! waylandsink sync=true fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/mobilenet_v2_1.0_224_quant.tflite ! queue ! qtimlvclassification threshold=60.0 results=3 module=mobilenet labels=/data/mobilenet.labels ! text/x-raw ! queue ! metamux.
          
        • 嵌入式Linux:
          export WAYLAND_DISPLAY=wayland-1 && export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/Animals_003_720p_180s_30FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtimetamux name=metamux ! queue ! qtioverlay ! queue ! waylandsink sync=true fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/mobilenet_v2_1.0_224_quant.tflite ! queue ! qtimlvclassification threshold=60.0 results=3 module=mobilenet labels=/data/mobilenet.labels ! text/x-raw ! queue ! metamux.
          
    • 显示(变体 2):使用qtivcomposer插件将原始帧与分类掩码混合。
      • 相机来源:
        • Linux Ubuntu:
          export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::position="<50, 50>" sink_1::dimensions="<368, 64>" ! queue ! waylandsink sync=false fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/mobilenet_v2_1.0_224_quant.tflite ! queue ! qtimlvclassification threshold=60.0 results=3 module=mobilenet labels=/data/mobilenet.labels ! video/x-raw,format=BGRA,width=368,height=64 ! queue ! mixer.
          
        • 嵌入式Linux:
          export WAYLAND_DISPLAY=wayland-1 && export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::position="<50, 50>" sink_1::dimensions="<368, 64>" ! queue ! waylandsink sync=false fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/mobilenet_v2_1.0_224_quant.tflite ! queue ! qtimlvclassification threshold=60.0 results=3 module=mobilenet labels=/data/mobilenet.labels ! video/x-raw,format=BGRA,width=368,height=64 ! queue ! mixer.
          
      • 文件来源:
        • Linux Ubuntu:
          export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/Animals_003_720p_180s_30FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::position="<50, 50>" sink_1::dimensions="<368, 64>" ! queue ! waylandsink sync=true fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/mobilenet_v2_1.0_224_quant.tflite ! queue ! qtimlvclassification threshold=60.0 results=3 module=mobilenet labels=/data/mobilenet.labels ! video/x-raw,format=BGRA,width=368,height=64 ! queue ! mixer.
          
        • 嵌入式Linux:
          export WAYLAND_DISPLAY=wayland-1 && export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/Animals_003_720p_180s_30FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::position="<50, 50>" sink_1::dimensions="<368, 64>" ! queue ! waylandsink sync=true fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/mobilenet_v2_1.0_224_quant.tflite ! queue ! qtimlvclassification threshold=60.0 results=3 module=mobilenet labels=/data/mobilenet.labels ! video/x-raw,format=BGRA,width=368,height=64 ! queue ! mixer.
          
    • 编码(变体 1):使用qtioverlay插件应用分类叠加。
      • 相机来源:
        gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtimetamux name=metamux ! queue ! qtioverlay ! queue ! qtic2venc target-bitrate=6000000 ! h264parse ! queue ! mp4mux ! queue ! filesink location=/data/video.mp4 split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/mobilenet_v2_1.0_224_quant.tflite ! queue ! qtimlvclassification threshold=60.0 results=3 module=mobilenet labels=/data/mobilenet.labels ! text/x-raw ! queue ! metamux.
        
      • 文件来源:
        gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/Animals_003_720p_180s_30FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtimetamux name=metamux ! queue ! qtioverlay ! queue ! qtic2venc target-bitrate=6000000 ! h264parse ! queue ! mp4mux ! queue ! filesink location=/data/video.mp4 split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/mobilenet_v2_1.0_224_quant.tflite ! queue ! qtimlvclassification threshold=60.0 results=3 module=mobilenet labels=/data/mobilenet.labels ! text/x-raw ! queue ! metamux.
        
    • 编码(变体 2):使用qtivcomposer插件将原始帧与分类掩码混合。
      • 相机来源:
        gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::position="<50, 50>" sink_1::dimensions="<368, 64>" ! queue ! qtic2venc target-bitrate=6000000 ! h264parse ! queue ! mp4mux ! queue ! filesink location=/data/video.mp4 split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/mobilenet_v2_1.0_224_quant.tflite ! queue ! qtimlvclassification threshold=60.0 results=3 module=mobilenet labels=/data/mobilenet.labels ! video/x-raw,format=BGRA,width=368,height=64 ! queue ! mixer.
        
      • 文件来源:
        gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/Animals_003_720p_180s_30FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::position="<50, 50>" sink_1::dimensions="<368, 64>" ! queue ! qtic2venc target-bitrate=6000000 ! h264parse ! queue ! mp4mux ! queue ! filesink location=/data/video.mp4 split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=hexagon model=/data/mobilenet_v2_1.0_224_quant.tflite ! queue ! qtimlvclassification threshold=60.0 results=3 module=mobilenet labels=/data/mobilenet.labels ! video/x-raw,format=BGRA,width=368,height=64 ! queue ! mixer.
        
  • Qualcomm 神经处理 SDK:执行用例的步骤

    • 显示(变体 1):使用qtioverlay插件应用分类叠加。
      • 相机来源:
        • Linux Ubuntu:
          export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtimetamux name=metamux ! queue ! qtioverlay text-font-size=24 ! queue ! waylandsink sync=false fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/mobilenet_v1_quantaware_quantized.dlc ! queue ! qtimlvclassification threshold=51.0 results=1 module=mobilenet labels=/data/mobilenet.labels ! text/x-raw ! queue ! metamux.
          
        • 嵌入式Linux:
          export WAYLAND_DISPLAY=wayland-1 && export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtimetamux name=metamux ! queue ! qtioverlay text-font-size=24 ! queue ! waylandsink sync=false fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/mobilenet_v1_quantaware_quantized.dlc ! queue ! qtimlvclassification threshold=51.0 results=1 module=mobilenet labels=/data/mobilenet.labels ! text/x-raw ! queue ! metamux.
          
      • 文件来源:
        • Linux Ubuntu:
          export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/Animals_003_720p_180s_30FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtimetamux name=metamux ! queue ! qtioverlay text-font-size=24 ! queue ! waylandsink sync=true fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/mobilenet_v1_quantaware_quantized.dlc ! queue ! qtimlvclassification threshold=51.0 results=1 module=mobilenet labels=/data/mobilenet.labels ! text/x-raw ! queue ! metamux.
          
        • 嵌入式Linux:
          export WAYLAND_DISPLAY=wayland-1 && export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/Animals_003_720p_180s_30FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtimetamux name=metamux ! queue ! qtioverlay text-font-size=24 ! queue ! waylandsink sync=true fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/mobilenet_v1_quantaware_quantized.dlc ! queue ! qtimlvclassification threshold=51.0 results=1 module=mobilenet labels=/data/mobilenet.labels ! text/x-raw ! queue ! metamux.
          
    • 显示(变体 2):使用qtivcomposer插件将原始帧与分类掩码混合。
      • 相机来源:
        • Linux Ubuntu:
          export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::position="<50, 50>" sink_1::dimensions="<368, 32>" ! queue ! waylandsink sync=false fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/mobilenet_v1_quantaware_quantized.dlc ! queue ! qtimlvclassification threshold=51.0 results=1 module=mobilenet labels=/data/mobilenet.labels ! video/x-raw,format=BGRA,width=368,height=32 ! queue ! mixer.
          
        • 嵌入式Linux:
          export WAYLAND_DISPLAY=wayland-1 && export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::position="<50, 50>" sink_1::dimensions="<368, 32>" ! queue ! waylandsink sync=false fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/mobilenet_v1_quantaware_quantized.dlc ! queue ! qtimlvclassification threshold=51.0 results=1 module=mobilenet labels=/data/mobilenet.labels ! video/x-raw,format=BGRA,width=368,height=32 ! queue ! mixer.
          
      • 文件来源:
        • Linux Ubuntu:

          export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/Animals_003_720p_180s_30FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::position="<50, 50>" sink_1::dimensions="<368, 32>" ! queue ! waylandsink sync=true fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/mobilenet_v1_quantaware_quantized.dlc ! queue ! qtimlvclassification threshold=51.0 results=1 module=mobilenet labels=/data/mobilenet.labels ! video/x-raw,format=BGRA,width=368,height=32 ! queue ! mixer.
          
        • 嵌入式Linux:

          export WAYLAND_DISPLAY=wayland-1 && export XDG_RUNTIME_DIR=/run/user/root && gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/Animals_003_720p_180s_30FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::position="<50, 50>" sink_1::dimensions="<368, 32>" ! queue ! waylandsink sync=true fullscreen=true split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/mobilenet_v1_quantaware_quantized.dlc ! queue ! qtimlvclassification threshold=51.0 results=1 module=mobilenet labels=/data/mobilenet.labels ! video/x-raw,format=BGRA,width=368,height=32 ! queue ! mixer.
          
    • 编码(变体 1):使用qtioverlay插件应用分类叠加。
      • 相机来源:
        gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtimetamux name=metamux ! queue ! qtioverlay text-font-size=24 ! queue ! qtic2venc target-bitrate=6000000 ! h264parse ! queue ! mp4mux ! queue ! filesink location=/data/video.mp4 split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/mobilenet_v1_quantaware_quantized.dlc ! queue ! qtimlvclassification threshold=51.0 results=1 module=mobilenet labels=/data/mobilenet.labels ! text/x-raw ! queue ! metamux.
        
      • 文件来源:
        gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/Animals_003_720p_180s_30FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtimetamux name=metamux ! queue ! qtioverlay text-font-size=24 ! queue ! qtic2venc target-bitrate=6000000 ! h264parse ! queue ! mp4mux ! queue ! filesink location=/data/video.mp4 split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/mobilenet_v1_quantaware_quantized.dlc ! queue ! qtimlvclassification threshold=51.0 results=1 module=mobilenet labels=/data/mobilenet.labels ! text/x-raw ! queue ! metamux.
        
    • 编码(变体 2):使用qtivcomposer插件将原始帧与分类掩码混合。
      • 相机来源:
        gst-launch-1.0 -e --gst-debug=2 qtiqmmfsrc name=camsrc ! video/x-raw\(memory:GBM\),format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::position="<50, 50>" sink_1::dimensions="<368, 32>" ! queue ! qtic2venc target-bitrate=6000000 ! h264parse ! queue ! mp4mux ! queue ! filesink location=/data/video.mp4 split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/mobilenet_v1_quantaware_quantized.dlc ! queue ! qtimlvclassification threshold=51.0 results=1 module=mobilenet labels=/data/mobilenet.labels ! video/x-raw,format=BGRA,width=368,height=32 ! queue ! mixer.
        
      • 文件来源:
        gst-launch-1.0 -e --gst-debug=2 filesrc location=/data/Animals_003_720p_180s_30FPS.mp4 ! qtdemux ! queue ! h264parse ! qtic2vdec ! queue ! tee name=split ! queue ! qtivcomposer name=mixer sink_1::position="<50, 50>" sink_1::dimensions="<368, 32>" ! queue ! qtic2venc target-bitrate=6000000 ! h264parse ! queue ! mp4mux ! queue ! filesink location=/data/video.mp4 split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/data/mobilenet_v1_quantaware_quantized.dlc ! queue ! qtimlvclassification threshold=51.0 results=1 module=mobilenet labels=/data/mobilenet.labels ! video/x-raw,format=BGRA,width=368,height=32 ! queue ! mixer.
        
  • 验证方法

    • 当应用程序收到 CTRL+ C并成功终止命令行工具时,会将 .mp4 文件保存到存储中。
    • 要验证 AI 处理,请检查显示屏上运行的视频流。应该有一个包含可能的分类标签的列表。
  • 24
    点赞
  • 10
    收藏
    觉得还不错? 一键收藏
  • 1
    评论
评论 1
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值