Stereo R-CNN解读

Stereo R-CNN

Stereo R-CNN based 3D Object Detection for Autonomous Driving

26 May 2019: Pytorch 1.0.0 and Python 3.6 are supported now. Please checkout to branch 1.0!
现在支持Pytorch 1.0.0和Python 3.6。请结帐到

This project contains the implementation of our CVPR 2019 paper arxiv.
这个项目包含了我们的CVPR 2019论文的实现

Stereo R-CNN focuses on accurate 3D object detection and estimation using image-only data in autonomous driving scenarios. It features simultaneous object detection and association for stereo images, 3D box estimation using 2D information, accurate dense alignment for 3D box refinement. We also provide a light-weight version based on the monocular 2D detection, which only uses stereo images in the dense alignment module. Please checkout to branch mono for details.

这个项目包含了我们的CVPR 2019论文的实现
Stereo R-CNN专注于在自动驾驶场景中使用纯图像数据进行精确的3D目标检测和估计。它的特点是同时对立体图像进行目标检测和关联，利用二维信息进行三维盒估计，精确的密集对齐进行三维盒细化。我们还提供了一个基于单目二维检测的轻量化版本，它只使用密集对齐模块中的立体图像。请到分行结账

Authors: Peiliang Li, Xiaozhi Chen and Shaojie Shen from the HKUST Aerial Robotics Group, and DJI.

If you find the project useful for your research, please cite:
如果你觉得这个项目对你的研究有用，请引用

@inproceedings{licvpr2019, 
    title     = {Stereo R-CNN based 3D Object Detection for Autonomous Driving}, 
    author    = {Li, Peiliang and Chen, Xiaozhi and Shen, Shaojie}, 
    booktitle = {CVPR}, 
    year      = {2019} 
}

0. Install

This implementation is tested under Pytorch 0.3.0. To avoid affecting your Pytorch version, we recommend using conda to enable multiple versions of Pytorch.
这个实现在Pytorch 0.3.0下进行了测试。为了避免影响Pytorch版本，我们建议使用conda来启用Pytorch的多个版本

0.0. Install Pytorch:

    conda create -n env_stereo python=2.7
    conda activate env_stereo
    conda install pytorch=0.3.0 cuda80 -c pytorch
    conda install torchvision -c pytorch

0.1. Other dependencies:
其他依赖项

    git clone git@github.com:HKUST-Aerial-Robotics/Stereo-RCNN.git
    cd stereo_rcnn
    pip install -r requirements.txt

0.2. Build:

    cd lib
    sh make.sh
    cd ..

1. Quick Demo

快速演示

1.0. Set the folder for placing the model
设置放置模型的文件夹

    mkdir models_stereo

1.1. Download our trained weight One Drive/Google Drive and put it into models_stereo/, then just run
下载我们的训练体重，把它放到模型里，然后就运行

    python demo.py

If everything goes well, you will see the detection result on the left, right and bird’s eye view image respectively.
如果一切顺利，你会看到检测结果的左，右和鸟瞰图分别。

2. Dataset Preparation

数据集准备

2.0. Download the left image, right image, calibration, labels and point clouds (optional for visualization) from KITTI Object Benchmark. Make sure the structure looks like:
从下载左图像、右图像、校准、标签和点云（可视化可选），确保结构看起来像：

yourfolder/object
              training
                image_2
                image_3
                label_2
                calib
                velodyne

2.1. Create symlinks:
创建符号链接

    cd data/kitti
    ln -s yourfolder/object object
    cd ../..

3. Training

Download the Res-101 pretrained weight One Drive/Google Drive, and put it into data/pretrained_model
下载Res-101预训练砝码，并将其放入数据/预训练模型中

Set corresponding CUDA_VISIBLE_DEVICES in train.sh, and simply run
在train.sh中设置相应的CUDA\u VISIBLE\u设备，只需运行

    ./train.sh

It consumes ~11G GPU memery for training. If your GPU memery is not enough, please try our light-weight version in branch mono.
它消耗~11G GPU内存用于训练。如果您的GPU内存不足，请尝试我们的轻量版本的分支单声道。

The trained model and training log are saved in /models_stereo by default.
默认情况下，训练模型和训练日志保存在/models\u中。

4. Evaluation

You can evaluate the 3D detection performance using either our provided model or your trained model.
Set corresponding CUDA_VISIBLE_DEVICES in test.sh, and run

您可以使用我们提供的模型或经过培训的模型来评估3D检测性能。
在test.sh中设置相应的CUDA\u VISIBLE\u设备，运行

    ./test.sh

The results are saved in models_stereo/result/data by default. You can evaluate the results using the tool from here.
默认情况下，结果保存在models/result/data中。您可以使用中的工具评估结果

Some sample results:

5. Acknowledgments

This repo is built based on the Faster RCNN implementation from faster-rcnn.pytorch and fpn.pytorch, and we also use the imagenet pretrained weight (originally provided from here) for initialization.
此回购是基于，我们还使用了imagenet预训练权重用于初始化。