多传感器融合学习笔记

目录

BevFusion

BEVFormer（ECCV 2022）

eagerMOT 利用2D+3D数据 多目标跟踪

Download 3D and 2D detections, which ones to download depends on what you want to run:

---

BevFusion

最小的模型530多m

BevFusion是一种多传感器融合技术，它可以将来自不同传感器（如LiDAR和相机）的数据融合到一个统一的BEV表示中。BevFusion的优点在于它能够结合多种传感器的优点，从而在物体检测和跟踪任务中取得更好的性能。

1. 相机端采用BEV算法生成BEV特征，如bevformer、LSS
2. LIDAR端用3D卷积提取BEV特征，与相机端特征concat起来。
3. 采用通道注意力，全局池化+1*1卷积表示相机端和LIDAR端特征的权重比。

BEVFormer（ECCV 2022）

模型1个G

论文链接：https://arxiv.org/pdf/2203.1727

项目：https://github.com/fundamentalvision/BEVFormer

后续：DETR，DETR3D

eagerMOT 利用2D+3D数据 多目标跟踪

if __name__ == "__main__":
    # choose which one to run, comment out the other one
    run_on_nuscenes()  
    run_on_kitti()

GitHub - aleksandrkim61/EagerMOT: Official code for "EagerMOT: 3D Multi-Object Tracking via Sensor Fusion" [ICRA 2021]

Download 3D and 2D detections, which ones to download depends on what you want to run:

We thank other researchers for opening their code and data to the community and therefore provide links to their data directly in hopes that you will also check out their great work! Data that was not available directly but was generated by us using their open source code is given through a cloud download link.

• KITTI 2D MOTSFusion detections/segmentations from GitHub - tobiasfshr/MOTSFusion: MOTSFusion: Track to Reconstruct and Reconstruct to Track. Under the "Results" section, they provide link to download their detections.
• KITTI 2D TrackRCNN detections/segmentations from Computer Vision. Under the "Downloads" section they provide a link to download detections.
• KITTI 3D PointGNN, NuScenes 3D CenterPoint, NuScenes 2D detections using an MMDetection model from the drive.
• NuScenes 3D CenterPoint detections can also be downloaded directly from the author's page for the centerpoint_voxel_1440_dcn(flip) config. NuScenes 2D MMDetection output was obtained using the cascade_mask_rcnn_x101 model given in the model zoo. See issue #9 for more details.
• KITTI 3D AB3DMOT detections can be downloaded from the original source GitHub - xinshuoweng/AB3DMOT: (IROS 2020, ECCVW 2020) Official Python Implementation for "3D Multi-Object Tracking: A Baseline and New Evaluation Metrics", but their format has recently changed and is no longer compatible with this repo. The detections provided in the above cloud link are simpy a copy downloaded at an earlier time when this the parsing code in this repo was written.