MonoGRNet
https://github.com/Zengyi-Qin/MonoGRNet
MonoGRNet: A Geometric Reasoning Network for Monocular 3D Object Localization
MonoGRNet:单张图像估测物体三维位置
MonoGRNet:一种用于单目三维物体检测和定位的几何推理网络
geometric [,dʒɪə'metrɪk]:adj. 几何学的,几何学图形的
reason ['riːz(ə)n]:n. 理由,理性,动机 vi. 推论,劝说 vt. 说服,推论,辩论
monocular [mə'nɒkjʊlə]:adj. 单眼的,单眼用的
localization [,ləʊkəlaɪ'zeɪʃən]:n. 定位,局限,地方化
Karlsruhe Institute of Technology and Toyota Technological Institute at Chicago,KITTI
Karlsruhe Institute of Technology (German: Karlsruher Institut für Technologie),KIT:卡尔斯鲁厄理工学院
Toyota Technological Institute at Chicago,TTIC or TTI-Chicago:芝加哥丰田科技学院
Toyota Technological Institute,TTI:丰田工业大学
1. MonoGRNet: A Geometric Reasoning Network for Monocular 3D Object Localization
amodal 3D bounding boxes,ABBox-3D:非模态三维边界框
instance depth estimation,IDE:实例深度估计,个体深度估计
instance-level depth estimation,IDE:个体级深度估计
local corner regression:局部角落回归
2. Prerequisites
- ubuntu 16.04 LTS
- Python 2.7
- Tensorflow 1.4.0
prerequisite [priː'rekwɪzɪt]:n. 先决条件 adj. 首要必备的
3. Installation
Clone this repository
git clone https://github.com/Zengyi-Qin/MonoGRNet.git
strong@foreverstrong:~/git_workspace$ git clone https://github.com/Zengyi-Qin/MonoGRNet
Cloning into 'MonoGRNet'...
remote: Enumerating objects: 135, done.
remote: Counting objects: 100% (135/135), done.
remote: Compressing objects: 100% (121/121), done.
remote: Total 135 (delta 38), reused 76 (delta 3), pack-reused 0
Receiving objects: 100% (135/135), 2.67 MiB | 1.11 MiB/s, done.
Resolving deltas: 100% (38/38), done.
Checking connectivity... done.
strong@foreverstrong:~/git_workspace$
strong@foreverstrong:~/git_workspace$ cd MonoGRNet/
strong@foreverstrong:~/git_workspace/MonoGRNet$ ls
data demo.jpg evals include LICENSE README.md submodules
decoder encoder hypes inputs optimizer setup.py train.py
strong@foreverstrong:~/git_workspace/MonoGRNet$
Download the KITTI Object Detection Dataset
(image
, calib
and label
) and place it into data/KittiBox
. The folder should be in the following structure:
data
KittiBox
training
calib
image_2
label_2
train.txt
val.txt
3D Object Detection Evaluation 2017
http://www.cvlibs.net/datasets/kitti/eval_object.php?obj_benchmark=3d
Download left color images of object data set (12 GB)
http://www.cvlibs.net/download.php?file=data_object_image_2.zip
spam folder:垃圾邮件文件夹
Download camera calibration matrices of object data set (16 MB)
http://www.cvlibs.net/download.php?file=data_object_calib.zip
Download training labels of object data set (5 MB)
http://www.cvlibs.net/download.php?file=data_object_label_2.zip
Download object development kit (1 MB) (including 3D object detection and bird’s eye view evaluation code)
devkit_object.zip
The train-val split train.txt
and val.txt
are contained in this repository.
Compile the Cython module and download the pretrained model:
python setup.py
strong@foreverstrong:~/git_workspace/MonoGRNet$ ll
total 320
drwxrwxr-x 13 strong strong 4096 May 30 14:06 ./
drwxrwxr-x 9 strong strong 4096 May 30 08:15 ../
drwxrwxr-x 2 strong strong 4096 May 30 14:18 3D Object Detection Evaluation 2017/
drwxrwxr-x 3 strong strong 4096 May 30 08:15 data/
drwxrwxr-x 2 strong strong 4096 May 30 08:15 decoder/
-rw-rw-r-- 1 strong strong 248497 May 30 08:15 demo.jpg
drwxrwxr-x 2 strong strong 4096 May 30 08:15 encoder/
drwxrwxr-x 2 strong strong 4096 May 30 08:15 evals/
drwxrwxr-x 8 strong strong 4096 May 30 08:15 .git/
drwxrwxr-x 2 strong strong 4096 May 30 08:15 hypes/
drwxrwxr-x 5 strong strong 4096 May 30 08:15 include/
drwxrwxr-x 2 strong strong 4096 May 30 08:15 inputs/
-rw-rw-r-- 1 strong strong 11356 May 30 08:15 LICENSE
drwxrwxr-x 2 strong strong 4096 May 30 08:15 optimizer/
-rw-rw-r-- 1 strong strong 2180 May 30 08:15 README.md
-rw-rw-r-- 1 strong strong 565 May 30 08:15 setup.py
drwxrwxr-x 3 strong strong 4096 May 30 08:15 submodules/
-rw-rw-r-- 1 strong strong 2401 May 30 08:15 train.py
strong@foreverstrong:~/git_workspace/MonoGRNet$
strong@foreverstrong:~/git_workspace/MonoGRNet$ python setup.py
pip install runcython
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/__init__.py:83: RequestsDependencyWarning: Old version of cryptography ([1, 2, 3]) may cause slowdown.
warnings.warn(warning, RequestsDependencyWarning)
Collecting runcython
Downloading https://files.pythonhosted.org/packages/e5/8c/14816b8df2b61660b26354545115f9b2fe3ceb032d775a111a08b15ec435/runcython-0.2.5.tar.gz
Requirement already satisfied: Cython>=0.10 in /usr/local/lib/python2.7/dist-packages (from runcython) (0.28.2)
Building wheels for collected packages: runcython
Running setup.py bdist_wheel for runcython ... done
Stored in directory: /home/strong/.cache/pip/wheels/77/73/f6/dbf7b8972de1de9c07361c5c2a9a1d2c02022e1168bd2a34b0
Successfully built runcython
Installing collected packages: runcython
Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/runcython-0.2.5.dist-info'
Consider using the `--user` option or check the permissions.
You are using pip version 18.0, however version 19.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Makefile:5: recipe for target 'all-python2' failed
make: *** [all-python2] Error 1
--2019-05-30 15:07:56-- https://cloud.tsinghua.edu.cn/f/d958b1da1c49496fa951/?dl=1
Resolving cloud.tsinghua.edu.cn (cloud.tsinghua.edu.cn)... 101.6.8.7
Connecting to cloud.tsinghua.edu.cn (cloud.tsinghua.edu.cn)|101.6.8.7|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://cloud.tsinghua.edu.cn/seafhttp/files/95f7ae29-bfd0-4b3e-86d5-824bdcfcc20e/data.tar.gz [following]
--2019-05-30 15:07:56-- https://cloud.tsinghua.edu.cn/seafhttp/files/95f7ae29-bfd0-4b3e-86d5-824bdcfcc20e/data.tar.gz
Reusing existing connection to cloud.tsinghua.edu.cn:443.
HTTP request sent, awaiting response... 200 OK
Length: 361164800 (344M) [application/octet-stream]
Saving to: ‘pretrained.tar.gz’
pretrained.tar.gz 32%[=========================> ] 112.92M 30.9KB/s in 2m 28s
2019-05-30 15:10:25 (780 KB/s) - Connection closed at byte 118406801. Retrying.
--2019-05-30 15:10:26-- (try: 2) https://cloud.tsinghua.edu.cn/seafhttp/files/95f7ae29-bfd0-4b3e-86d5-824bdcfcc20e/data.tar.gz
Connecting to cloud.tsinghua.edu.cn (cloud.tsinghua.edu.cn)|101.6.8.7|:443... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 361164800 (344M), 242757999 (232M) remaining [application/octet-stream]
Saving to: ‘pretrained.tar.gz’
pretrained.tar.gz 100%[++++++++++++++++++++++++++=====>] 344.43M 2.75MB/s in 1m 56s
2019-05-30 15:12:23 (1.99 MB/s) - ‘pretrained.tar.gz’ saved [361164800/361164800]
Pretrained model downloaded.
Setup finished!
strong@foreverstrong:~/git_workspace/MonoGRNet$
#!/usr/bin/env python3
# -*- coding:utf-8 -*-
import os
# Cython module
os.system("cd ./include/utils/ && make")
# download the pre-trained model
if not os.path.exists("./data/model_2D.pkl"):
os.system("wget https://cloud.tsinghua.edu.cn/f/d958b1da1c49496fa951/?dl=1 -O pretrained.tar.gz")
os.system("tar -xf pretrained.tar.gz")
os.system("rm pretrained.tar.gz")
print("pretrained model")
# 3D Object Detection Evaluation 2017
if not os.path.exists("./data/KittiBox/training"):
print("KITTI dataset was not found in ./data/KittiBox/training.")
exit("ERROR!")
print("SUCCESS!")
python3 setup.py
3.1 Makefile
SHELL := /bin/bash
.PHONY : all-python2 all-python3
all-python2:
sudo pip install runcython
makecython++ stitch_wrapper.pyx "" "stitch_rects.cpp ./hungarian/hungarian.cpp"
all-python3:
sudo pip3 install runcython3
makecython3++ stitch_wrapper.pyx "" "stitch_rects.cpp ./hungarian/hungarian.cpp"
make all-python2 - make all-python3 - make
strong@foreverstrong:~/git_workspace/MonoGRNet/include/utils$ cat Makefile
SHELL := /bin/bash
.PHONY : all-python2 all-python3
all-python2:
sudo pip install runcython
makecython++ stitch_wrapper.pyx "" "stitch_rects.cpp ./hungarian/hungarian.cpp"
all-python3:
sudo pip3 install runcython3
makecython3++ stitch_wrapper.pyx "" "stitch_rects.cpp ./hungarian/hungarian.cpp"
strong@foreverstrong:~/git_workspace/MonoGRNet/include/utils$
strong@foreverstrong:~/git_workspace/MonoGRNet/include/utils$ make
sudo pip install runcython
[sudo] password for strong:
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/__init__.py:83: RequestsDependencyWarning: Old version of cryptography ([1, 2, 3]) may cause slowdown.
warnings.warn(warning, RequestsDependencyWarning)
The directory '/home/strong/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/strong/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting runcython
Requirement already satisfied: Cython>=0.10 in /usr/local/lib/python2.7/dist-packages (from runcython) (0.28.2)
Installing collected packages: runcython
Successfully installed runcython-0.2.5
makecython++ stitch_wrapper.pyx "" "stitch_rects.cpp ./hungarian/hungarian.cpp"
strong@foreverstrong:~/git_workspace/MonoGRNet/include/utils$
strong@foreverstrong:~/git_workspace/MonoGRNet/include/utils$ make all-python2
sudo pip install runcython
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/__init__.py:83: RequestsDependencyWarning: Old version of cryptography ([1, 2, 3]) may cause slowdown.
warnings.warn(warning, RequestsDependencyWarning)
The directory '/home/strong/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/strong/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Requirement already satisfied: runcython in /usr/local/lib/python2.7/dist-packages (0.2.5)
Requirement already satisfied: Cython>=0.10 in /usr/local/lib/python2.7/dist-packages (from runcython) (0.28.2)
makecython++ stitch_wrapper.pyx "" "stitch_rects.cpp ./hungarian/hungarian.cpp"
strong@foreverstrong:~/git_workspace/MonoGRNet/include/utils$
strong@foreverstrong:~/git_workspace/MonoGRNet/include/utils$ make all-python3
sudo pip3 install runcython3
The directory '/home/strong/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/strong/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting runcython3
Downloading https://files.pythonhosted.org/packages/51/9f/7bf25cc3daa53472b3afe58742c218138c70a4a0d6af2cdf13d0ff908a4a/runcython3-0.2.7.tar.gz
Requirement already satisfied (use --upgrade to upgrade): Cython>=0.10 in /usr/local/lib/python3.5/dist-packages (from runcython3)
Installing collected packages: runcython3
Running setup.py install for runcython3 ... done
Successfully installed runcython3-0.2.7
You are using pip version 8.1.1, however version 19.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
makecython3++ stitch_wrapper.pyx "" "stitch_rects.cpp ./hungarian/hungarian.cpp"
strong@foreverstrong:~/git_workspace/MonoGRNet/include/utils$
strong@foreverstrong:~/git_workspace/MonoGRNet/include/utils$ ll
total 192
drwxrwxr-x 5 strong strong 4096 Jun 3 10:38 ./
drwxrwxr-x 5 strong strong 4096 May 30 08:15 ../
drwxrwxr-x 3 strong strong 4096 May 30 08:15 annolist/
-rw-rw-r-- 1 strong strong 9163 May 30 08:15 data_utils.py
drwxrwxr-x 2 strong strong 4096 May 30 08:15 hungarian/
-rw-rw-r-- 1 strong strong 0 May 30 08:15 __init__.py
drwxrwxr-x 2 strong strong 4096 May 30 08:15 kaffe/
-rw-rw-r-- 1 strong strong 302 Jun 3 10:37 Makefile
-rw-rw-r-- 1 strong strong 1936 May 30 08:15 rect.py
-rw-rw-r-- 1 strong strong 3798 May 30 08:15 stitch_rects.cpp
-rw-rw-r-- 1 strong strong 3177 May 30 08:15 stitch_rects.hpp
-rw-rw-r-- 1 strong strong 117 May 30 08:15 stitch_wrapper.py
-rw-rw-r-- 1 strong strong 3802 May 30 08:15 stitch_wrapper.pyx
-rwxrwxr-x 1 strong strong 118448 Jun 3 10:38 stitch_wrapper.so*
-rw-rw-r-- 1 strong strong 1063 May 30 08:15 TENSORBOX_LICENSE.txt
-rw-rw-r-- 1 strong strong 13396 May 30 08:15 train_utils.py
strong@foreverstrong:~/git_workspace/MonoGRNet/include/utils$ make all-python2
sudo pip install runcython
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/__init__.py:83: RequestsDependencyWarning: Old version of cryptography ([1, 2, 3]) may cause slowdown.
warnings.warn(warning, RequestsDependencyWarning)
The directory '/home/strong/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/strong/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Requirement already satisfied: runcython in /usr/local/lib/python2.7/dist-packages (0.2.5)
Requirement already satisfied: Cython>=0.10 in /usr/local/lib/python2.7/dist-packages (from runcython) (0.28.2)
makecython++ stitch_wrapper.pyx "" "stitch_rects.cpp ./hungarian/hungarian.cpp"
strong@foreverstrong:~/git_workspace/MonoGRNet/include/utils$ ll
total 192
drwxrwxr-x 5 strong strong 4096 Jun 3 10:44 ./
drwxrwxr-x 5 strong strong 4096 May 30 08:15 ../
drwxrwxr-x 3 strong strong 4096 May 30 08:15 annolist/
-rw-rw-r-- 1 strong strong 9163 May 30 08:15 data_utils.py
drwxrwxr-x 2 strong strong 4096 May 30 08:15 hungarian/
-rw-rw-r-- 1 strong strong 0 May 30 08:15 __init__.py
drwxrwxr-x 2 strong strong 4096 May 30 08:15 kaffe/
-rw-rw-r-- 1 strong strong 302 Jun 3 10:37 Makefile
-rw-rw-r-- 1 strong strong 1936 May 30 08:15 rect.py
-rw-rw-r-- 1 strong strong 3798 May 30 08:15 stitch_rects.cpp
-rw-rw-r-- 1 strong strong 3177 May 30 08:15 stitch_rects.hpp
-rw-rw-r-- 1 strong strong 117 May 30 08:15 stitch_wrapper.py
-rw-rw-r-- 1 strong strong 3802 May 30 08:15 stitch_wrapper.pyx
-rwxrwxr-x 1 strong strong 118160 Jun 3 10:44 stitch_wrapper.so*
-rw-rw-r-- 1 strong strong 1063 May 30 08:15 TENSORBOX_LICENSE.txt
-rw-rw-r-- 1 strong strong 13396 May 30 08:15 train_utils.py
strong@foreverstrong:~/git_workspace/MonoGRNet/include/utils$
4. Training and evaluation
Run the training script and specify the GPU to use:
python train.py --gpus 0
The evaluation is done during training. You can adjust the evaluation intervals in hypes/kittiBox.json
.