MobiSys 2023 | Fusang: Graph-inspired Robust and Accurate Object Recognition on Commodity mmWave Dev

原文链接：https://mp.weixin.qq.com/s?__biz=Mzg4MjgxMjgyMg==&mid=2247486272&idx=1&sn=045a55db92ff5e6c5c088c21de37cc61&chksm=cf51b9b9f82630af14c3e0f56d313d1c46136d67f4879f3f666106bf66bfcf02d883334318c0#rd

MobiSys 2023 | Fusang: Graph-inspired Robust and Accurate Object Recognition on Commodity mmWave Devices

无线感知论文阅读笔记 | MobiSys 2023 Fusang: Graph-inspired Robust and Accurate Object Recognition on Commodity mmWave Devices

Abstract

• Fusang
  • Proposes a low-barrier 3D object recognition system for COTS mmWave devices
  • Leverages mmWave large bandwidth to capture fine-grained reflected responses from object shapes
• features
  • Constructs two graph-structured features to represent reflected responses in frequency and IQ domain
  • network: accurately recognize objects even in multipath scenarios
• prototype
  • Implements a Fusang prototype on a commodity mmWave Radar device
  • Experiments show 97% mean accuracy on 24 objects in multipath environments

Open Srouce: https://github.com/OpenNISLab/Pro-Fusang

1 Introduction

• Motivation & Problem

  • Object recognition 很重要
  • 光学摄像头受多种限制
  • 毫米波成像分辨率低下难以准确识别

• Proposed Solution

  • 提出Fusang,仅依赖商用毫米波雷达实现高分辨三维目标识别

  • 1 HRRP: 建立目标散射点在视线方向上的投影,即高分辨距离剖面(HRRP)

    ✅ $\Rightarrow$ 通过在每个距离单元中对来自目标散射体的复杂时间回波进行相干求和而获得的一维向量,直接将目标轮廓与雷达信号的散射特征相关联

    ❌ 缺点：室内多径的影响 + 目标的HRRP数据本身就容易受到不同参数和测量条件的影响

  • 2 使用SVMD和定位捕获多径环境下信号的多频信息 $\Rightarrow$ 尽管目标点的频率成分可能有不同的幅值,但其能量分布保持稳定且独特

  • 3 开发树状模型和IQ信号特征稳健表征轮廓和曲率 $\Rightarrow$ 新的树状图结构来表示目标散射体的能量分布,提高HRRP数据的鲁棒性 + 记录来自多个天线在不同角度下反射信号的IQ样本,并在IQ域中设计定制特征以实现细粒度的曲率检测

  • 4 设计网络实现准确三维目标识别

• Results

  • Prototype and verify Fusang performance on 24 common objects

    ✅ Recognition accuracy up to 99% and no less than 88%.

  • Outperform prior HRRP-based method by 1.5× under the same SNR.

  • Verify robustness to environments and motions.

• Contributions :

  • 1 Fusang, Low-barrier 3D recognition system
  • 2 signal reconstruction for multipath environments: 将距离谱与SVMD相结合,以捕获目标信号的多频分量
  • 3 robust profile and curvature characterization: 树状模型来捕获雷达HRRP数据和IQ信号的稳定能量分布
  • 4 accurate 3D recognition network

2 BACKGROUND AND MOTIVATION

• 2.1 介绍雷达HRRP数据的生成原理
• 2.2 分析为什么它不能直接用于目标的三维建模
• 2.3 提出了关于HRRP能量分布的重要观察
  • 这启发了Fusang的设计

2.1 Review of Radar HRRP Data

• HRRP：目标散射点在视线方向上的投影
• 通过FFT提取HRRP
  • $\begin{array}{rl}& hrrp=[|hrrp(1)|,|hrrp(2)|,\dots ,|hrrp(i)|]\\ & =\left[\left|\sum _{r\in R_1}{\sigma }_r{e}^{j4\pi \left(f_c+Kt\right)d_{ra}/c}\right|,\dots \left|\sum _{r\in R_i}{\sigma }_r{e}^{j4\pi \left(f_c+Kt\right)d_{ra}/c}\right|\right]\end{array}$

2.2 Limitation of Using HRRP Directly

• HRRP能够识别不同物体的原理：

  • 不同的物体,它们的结构差异导致散射体幅度的差异 $\Rightarrow$ 最终表现为HRRP的区别
  • 可利用HRRP识别未知目标的类别

• HRRP挑战1：HRRP数据的脆弱性

  • 对于同一目标和同一天线,在连续时刻同一位置的每个FFT频点的峰值也会不同
  • 幅值的敏感性差异使得直接匹配参考数据与测量数据变得困难
  • $\Rightarrow$ 难以直接用于物体识别

• HRRP挑战2：类似轮廓的目标HRRP难以区分

2.3 Energy Distribution of Reflected Signal

• HRRP频谱能量分布稳定可利用
  • 同一目标的HRRP数据具有相似的能量分布(相关系数≥0.93)
  • 而不同目标的能量分布显着不相同(相关系数≤0.4)
• IQ spatial distribution 反映曲率信息,可区分形状相似目标
  • 不同曲率的物体以不同程度散射电磁波
  • 当雷达信号以不同的空间角度照射目标时,反射信号的IQ样本分布呈现周期性变化
  • 根据物体曲率与IQ信号之间的关系,可以识别一些形状相似但曲率不同的物体

3 FUSANG DESIGN

3.1 Overview

• Step 1 Multipath Signal Processing : 多径信号处理模块去噪原始毫米波信号
• Step 2 HRRP Graphical Representation : 构建称为“树枝”的二叉树图表示,稳健表达HRRP能量分布
• Step 3 Curvature Feature Extraction : 设计“叶子”特征,从不同角度的IQ信号中提取准确的曲率特征
• Step 4 3D Object Recognition : 构建包含GCN、LSTM和融合模块的混合神经网络,实现准确的三维目标识别

3.2 Multipath Signal Processing

• 目标定位利用距离谱和CFAR获得目标位置和频率分量
• SVMD算法分解重构信号,消除不在目标频率的背景噪声
  • SVMD： min ⁡ { u κ } , { ω κ } { ∑ κ ∥ ∂ t [ ( δ ( t ) + j π t ) ∗ u κ ( t ) ] e − j ω κ t ∥ 2 2 } \min _{\left\{u_\kappa\right\},\left\{\omega_\kappa\right\}}\left\{\sum_\kappa\left\|\partial_t\left[\left(\delta(t)+\frac{j}{\pi t}\right) * u_\kappa(t)\right] e^{-j \omega_\kappa t}\right\|_2^2\right\} min{uκ​},{ωκ​}​{∑κ​ ​∂t​[(δ(t)+πtj​)∗uκ​(t)]e−jωκ​t ​22​}
  • filtered signal: S b ( t ) = S ( t ) − ∑ κ u κ ( t ) , f b ∉ { ω κ } S_b(t)=S(t)-\sum_\kappa u_\kappa(t), \quad f_b \notin\left\{\omega_\kappa\right\} Sb​(t)=S(t)−∑κ​uκ​(t),fb​∈/{ωκ​}

3.3 HRRP Graphical Representation

• 提取HRRP包络的局部最大值 formants
• 迭代二分频谱,用 formants 作为节点构建二叉树表示轮廓
• 优点
  • 对HRRP幅值变化具有鲁棒性
  • 不依赖数据先验知识

3.4 Curvature Feature Extraction

• 观察到IQ信号展现出与表面曲率对应的不同分布

  

• 基于IQ样本密度统计设计“叶子”特征表征曲率差异

  • Leaves $(\theta )=\frac{{\mathrm{Num}}_{IQ}}{{\mathrm{area}}_{\theta \in \Psi }\left(\alpha ,{\mathcal{H}}^{\star }A(\theta )\right)}$
    

3.5 3D Object Recognition

• GCN分类器 捕获“树枝”的结构属性,实现稳健的轮廓识别
• LSTM分类器 建模“叶子”多角度下的时间依赖性,实现细粒度曲率
• 融合模块 组合两个分类器的预测,确定三维类别

4 IMPLEMENTATION

• Hardware platform
  • TI IWR1843 BOOST mmWave radar and data acquisition board
  • 3 TX 4 RX antennas, 77GHz 3.99GHz bandwidth
• Software implementation
  • Matlab processes radar signals
  • Pytorch builds neural network
• Data collection

  • 24 daily objects, signals collected at rotations

    ✅ 对于每个物体,顺时针旋转每个物体,并在9个角度(从0°到180°,每次20°)处收集反射信号, range: 1-5米

    ✅ 不可否认,不同的视角和距离在我们系统的训练过程中都是必需的

    ✅ 然而,并非所有距离和角度都是必要的

    ✅ 事实上,实验表明,当目标距离和角度略有变化时,从目标HRRP构造的图结构特征之间的差异并不明显

  • 14,000 frames total for training and testing

• Model training
  • GCN: 1000 epochs, Adam optimizer
  • LSTM: RMSprop optimizer

5 EVALUATION

5.1 Overall Performance

• 24个日常目标,与2个基准对比
  

• Fusang平均识别准确率98.2%,优于基准
  

5.2 Robustness

• Accuracy above 89% under distance variation, outperforms baselines

• Improves overall accuracy by 30% and 15% under angle variation

  

• Average accuracy above 96% across 3 environments

5.3 Dynamic Case Study

• 不同速度下准确率84-100%
• 办公室略低于研讨室
  • Doppler效应和多路径影响

6 DISCUSSION AND LIMITATION

3个limitation: 扫描、系统延迟、小目标和未知目标

• Scanning angle
  • 仍需少量运动雷达在训练期间从不同角度提供雷达数据集
  • 但Complexity greatly reduced compared to SAR methods
  • 96% accuracy with 2 angles training
• System delay
  • Currently non-real-time, main reason is long multipath signal processing
  • Real-time feasible through code optimization
• Small size and unseen objects
  • Sparse HRRP data for sizes below resolution, but IQ distribution usable
  • Currently cannot recognize unseen objects, expandable via generated data

7 RELATED WORK

• Camera and Lidar based
  • Performance affected by environment, privacy invasive
• RF based
  • Complex devices, hard to enable fine sensing
• mmWave based
  • Require mechanical motion or other devices
• HRRP based
  • 统计建模依赖先验,泛化能力差
  • ML方法需要大量标注样本
• Fusang system
  • Key 1: Graphical representation improves feature stability
  • Key 2: IQ combined improves sample efficiency

8 CONCLUSIONS

• Fusang system
  • Single-chip mmWave radar
  • No need for target augmentation or specialized optical hardware
  • Graph features improve HRRP stability + IQ curvature extraction
• Validation
  • 24 objects, multipath scenarios
  • Excellent accuracy and robustness