【复现笔记】SRT3D: Sparse Region-Based 3D Object Tracking

已于 2023-01-15 00:24:11 修改
阅读量1.2k 收藏 10
点赞数
分类专栏： 6D姿态估计 文章标签： 深度学习
于 2022-10-30 19:47:17 首次发布
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/weixin_54470372/article/details/127528451
版权

paper: SRT3D: A Sparse Region-Based 3D Object Tracking Approach for the Real World
Manuel Stoiber, Martin Pfanne, Klaus H. Strobl, Rudolph Triebel, and Alin Albu-Schäffer
International Journal of Computer Vision (IJCV) 2022

作者提供源代码：3DObjectTracking/SRT3D at master · DLR-RM/3DObjectTracking · GitHub

目录

目录

Description of Content

USAGE

开始复现

1、cmake

2、build

Description of Content

This repository contains everything necessary to reproduce the results presented in our paper. This includes the evaluation on the RBOT and OPT datasets as well as the conducted parameters study. Source files for the srt3d library are stored in src and include/srt3d. Source files for the executables are stored in examples. For the evaluation and parameter study on the RBOT and OPT datasets, the code in evaluate_rbot_dataset.cpp, parameters_study_rbot.cpp, evaluate_opt_dataset.cpp, and parameters_study_opt.cpp was used. To reproduce our experiments, please download the RBOT and OPT datasets and adjust the dataset_directory in the source code. Note that model files (e.g. ape_model.bin, bakingsoda_model.bin, ...) will be created automatically and are stored in the same folder as the .obj file of each object.

For real-world tracking, we provide code that allows either to directly track objects or to first record a sequence and then run the tracker on this recorded sequence. The code for this is provided in run_on_camera_sequence.cpp, record_camera_sequence.cpp and run_on_recorded_sequence.cpp. In our implementation, an Azure Kinect camera is used. If you don't want to use the Azure Kinect, you can disable the CMake option USE_AZURE_KINECT. For tracking, object parameters and directories in run_on_camera_sequence.cpp and run_on_recorded_sequence.cpp have to be adjusted. Note that by default, tracking will not start automatically. To start tracking, please press the T key on your keyboard. To stop the application press Q. If you would like to use another camera than the Azure Kinect, we encourage you to create a class similar to the AzureKinectCamera class in src/azure_kinect_camera.cpp.

 翻译：

这个存储库包含再现我们论文中给出的结果所需的所有内容。这包括对RBOT和OPT数据集的评价以及进行的参数研究。srt3d库的源文件存储在src中，包括/srt3d。可执行文件的源文件存储在示例中。对于RBOT和OPT数据集的评估和参数研究，使用了evaluate_rbot_dataset.cpp、parameters_study_rbot.cpp、evaluate_opt_dataset.cpp和parameters_study_opt.cpp中的代码。为了再现我们的实验，请下载RBOT和OPT数据集，并调整源代码中的dataset_directory。注意，模型文件(例如ape_model.bin, bakingsoda_model.bin，…)将被自动创建，并与每个对象的.obj文件存储在同一文件夹中。

对于真实世界的跟踪，我们提供的代码既可以直接跟踪对象，也可以先记录一个序列，然后在这个记录的序列上运行跟踪器。这方面的代码在run_on_camera_sequence.cpp、record_camera_sequence.cpp和run_on_recorded_sequence.cpp中提供。在我们的实现中，使用了Azure Kinect摄像头。如果你不想使用Azure Kinect，你可以禁用CMake选项USE_AZURE_KINECT。为了跟踪，必须调整run_on_camera_sequence.cpp和run_on_recorded_sequence.cpp中的对象参数和目录。注意，默认情况下，跟踪不会自动启动。要开始追踪，请按键盘上的T键。如果你想使用AzureKinect以外的其他摄像头，我们建议你在src/azure_kinect_camera.cpp中创建一个类似于AzureKinectCamera类的类。

USAGE

If you want to use our tracker for your own project, we would like to refer you to run_on_camera_sequence.cpp for how to set up the tracker and required objects. The following variables in run_on_camera_sequence.cpp should be considered in an initial set up:

  • model_path: location where all object models, that are created automatically from an .obj file, are stored.
  • body_ptr: contains all information associated with an object
    • geometry_path: path to .obj file
    • geometry2body_pose: transformation that allows to set a different frame of reference for the object than defined by the .obj file.
    • world2body_pose: initial transformation between camera and object.
    • geometry_unit_in_meter: scale factor to scale the unit used in the .obj file to meter.
    • geometry_counterclockwise: true if winding order of triangles in .obj is defined counter-clockwise.
    • geometry_enable_culling: true if faces that are not facing toward the camera should be culled.
    • maximum_body_diameter: maximum diameter in meter of a sphere that encapsulates the entire body.
    • occlusion_mask_id: unique number between 0 and 7 that is used to encode the object in occlusion_masks.
  • model_ptr: contains all information associated with a model
    • sphere_radius: distance from camera to object center
    • n_divides: how often an icosahedron is devided (controls the number of template views)
    • n_points: number of contour points

翻译：

如果您想在自己的项目中使用我们的跟踪器，我们建议您参考run_on_camera_sequence.cpp，了解如何设置跟踪器和所需的对象。

在初始设置中应该考虑run_on_camera_sequence.cpp中的以下变量:

  • Model_path:存储从.obj文件自动创建的所有对象模型的位置。
  • Body_ptr:包含与对象相关的所有信息
  • Geometry_path: .obj文件的路径
  • Geometry2body_pose:转换，允许为对象设置不同的参考框架，而不是由.obj文件定义的。
  • World2body_pose:相机和物体之间的初始转换。
  • Geometry_unit_in_meter:缩放因子，将.obj文件中使用的单位缩放为米。
  • geometry_逆时针:如果.obj中三角形的缠绕顺序定义为逆时针，则为true。
  • Geometry_enable_culling:如果不面向相机的脸应该被剔除，则为true。
  • Maximum_body_diameter:封装整个身体的球体的最大直径(米)。
  • Occlusion_mask_id: 0到7之间的唯一数字，用于编码occlusion_masks中的对象。
  • Model_ptr:包含与模型相关的所有信息
  • Sphere_radius:摄像机到物体中心的距离
  • n_partitions:划分二十面体的频率(控制模板视图的数量)
  • N_points:轮廓点个数

开始复现

1、编译

（1）cmake

cmake -DUSE_AZURE_KINECT=OFF

输出：

~/3dTracking/3DObjectTracking/SRT3D$ cmake -DUSE_AZURE_KINECT=OFF
CMake Warning:
  No source or binary directory provided.  Both will be assumed to be the
  same as the current working directory, but note that this warning will
  become a fatal error in future CMake releases.


-- The C compiler identification is GNU 9.4.0
-- The CXX compiler identification is GNU 9.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Warning (dev) at /usr/local/share/cmake-3.25/Modules/FindOpenGL.cmake:315 (message):
  Policy CMP0072 is not set: FindOpenGL prefers GLVND by default when
  available.  Run "cmake --help-policy CMP0072" for policy details.  Use the
  cmake_policy command to set the policy and suppress this warning.

  FindOpenGL found both a legacy GL library:

    OPENGL_gl_LIBRARY: /usr/lib/x86_64-linux-gnu/libGL.so

  and GLVND libraries for OpenGL and GLX:

    OPENGL_opengl_LIBRARY: /usr/lib/x86_64-linux-gnu/libOpenGL.so
    OPENGL_glx_LIBRARY: /usr/lib/x86_64-linux-gnu/libGLX.so

  OpenGL_GL_PREFERENCE has not been set to "GLVND" or "LEGACY", so for
  compatibility with CMake 3.10 and below the legacy GL library will be used.
Call Stack (most recent call first):
  CMakeLists.txt:22 (find_package)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Found OpenGL: /usr/lib/x86_64-linux-gnu/libOpenGL.so   
-- Found GLEW: /usr/include (found version "2.1.0") 
-- Found OpenCV: /usr/local (found suitable version "4.0.0", minimum required is "4.0.0") 
-- Found OpenMP_C: -fopenmp (found version "4.5") 
-- Found OpenMP_CXX: -fopenmp (found version "4.5") 
-- Found OpenMP: TRUE (found version "4.5")  
-- Performing Test HAS_MARCH
-- Performing Test HAS_MARCH - Failed
-- Performing Test HAS_MTUNE
-- Performing Test HAS_MTUNE - Failed
-- Performing Test HAS_GGDB
-- Performing Test HAS_GGDB - Success
-- Performing Test HAS_Z7
-- Performing Test HAS_Z7 - Failed
-- Performing Test HAS_FTRAPV
-- Performing Test HAS_FTRAPV - Success
-- Performing Test HAS_OD
-- Performing Test HAS_OD - Failed
-- Performing Test HAS_OB3
-- Performing Test HAS_OB3 - Failed
-- Performing Test HAS_O3
-- Performing Test HAS_O3 - Success
-- Performing Test HAS_OB2
-- Performing Test HAS_OB2 - Failed
-- Performing Test HAS_O2
-- Performing Test HAS_O2 - Success
-- Configuring done
-- Generating done
-- Build files have been written to: /home/r*/3dTracking/3DObjectTracking/SRT3D

（2）make

make

输出：

~/3dTracking/3DObjectTracking/SRT3D$ make
[  3%] Building CXX object src/CMakeFiles/srt3d.dir/common.cpp.o
[  6%] Building CXX object src/CMakeFiles/srt3d.dir/body.cpp.o
[ 10%] Building CXX object src/CMakeFiles/srt3d.dir/renderer_geometry.cpp.o
[ 13%] Building CXX object src/CMakeFiles/srt3d.dir/renderer.cpp.o
[ 17%] Building CXX object src/CMakeFiles/srt3d.dir/normal_renderer.cpp.o
[ 20%] Building CXX object src/CMakeFiles/srt3d.dir/occlusion_renderer.cpp.o
[ 24%] Building CXX object src/CMakeFiles/srt3d.dir/model.cpp.o
[ 27%] Building CXX object src/CMakeFiles/srt3d.dir/camera.cpp.o
[ 31%] Building CXX object src/CMakeFiles/srt3d.dir/loader_camera.cpp.o
[ 34%] Building CXX object src/CMakeFiles/srt3d.dir/viewer.cpp.o
[ 37%] Building CXX object src/CMakeFiles/srt3d.dir/image_viewer.cpp.o
/home/r*/3dTracking/3DObjectTracking/SRT3D/src/image_viewer.cpp: In member function ‘virtual bool srt3d::ImageViewer::UpdateViewer(int)’:
/home/r*/3dTracking/3DObjectTracking/SRT3D/src/image_viewer.cpp:30:1: warning: control reaches end of non-void function [-Wreturn-type]
   30 | }
      | ^
[ 41%] Building CXX object src/CMakeFiles/srt3d.dir/normal_viewer.cpp.o
/home/r*/3dTracking/3DObjectTracking/SRT3D/src/normal_viewer.cpp: In member function ‘virtual bool srt3d::NormalViewer::UpdateViewer(int)’:
/home/r*/3dTracking/3DObjectTracking/SRT3D/src/normal_viewer.cpp:60:1: warning: control reaches end of non-void function [-Wreturn-type]
   60 | }
      | ^
[ 44%] Building CXX object src/CMakeFiles/srt3d.dir/region_modality.cpp.o
[ 48%] Building CXX object src/CMakeFiles/srt3d.dir/tracker.cpp.o
[ 51%] Linking CXX static library libsrt3d.a
[ 51%] Built target srt3d
[ 55%] Building CXX object examples/CMakeFiles/evaluate_rbot_dataset.dir/evaluate_rbot_dataset.cpp.o
[ 58%] Building CXX object examples/CMakeFiles/evaluate_rbot_dataset.dir/rbot_evaluator.cpp.o
[ 62%] Linking CXX executable evaluate_rbot_dataset
[ 62%] Built target evaluate_rbot_dataset
[ 65%] Building CXX object examples/CMakeFiles/parameter_study_rbot_dataset.dir/parameter_study_rbot_dataset.cpp.o
[ 68%] Building CXX object examples/CMakeFiles/parameter_study_rbot_dataset.dir/rbot_evaluator.cpp.o
[ 72%] Linking CXX executable parameter_study_rbot_dataset
[ 72%] Built target parameter_study_rbot_dataset
[ 75%] Building CXX object examples/CMakeFiles/evaluate_opt_dataset.dir/evaluate_opt_dataset.cpp.o
[ 79%] Building CXX object examples/CMakeFiles/evaluate_opt_dataset.dir/opt_evaluator.cpp.o
[ 82%] Linking CXX executable evaluate_opt_dataset
[ 82%] Built target evaluate_opt_dataset
[ 86%] Building CXX object examples/CMakeFiles/parameter_study_opt_dataset.dir/parameter_study_opt_dataset.cpp.o
[ 89%] Building CXX object examples/CMakeFiles/parameter_study_opt_dataset.dir/opt_evaluator.cpp.o
[ 93%] Linking CXX executable parameter_study_opt_dataset
[ 93%] Built target parameter_study_opt_dataset
[ 96%] Building CXX object examples/CMakeFiles/run_on_recorded_sequence.dir/run_on_recorded_sequence.cpp.o
[100%] Linking CXX executable run_on_recorded_sequence
[100%] Built target run_on_recorded_sequence

（3）切换到examples目录

（4）在examples中cmake

~/3dTracking/3DObjectTracking/SRT3D/examples$ cmake ..
CMake Warning (dev) at /usr/local/share/cmake-3.25/Modules/FindOpenGL.cmake:315 (message):
  Policy CMP0072 is not set: FindOpenGL prefers GLVND by default when
  available.  Run "cmake --help-policy CMP0072" for policy details.  Use the
  cmake_policy command to set the policy and suppress this warning.

  FindOpenGL found both a legacy GL library:

    OPENGL_gl_LIBRARY: /usr/lib/x86_64-linux-gnu/libGL.so

  and GLVND libraries for OpenGL and GLX:

    OPENGL_opengl_LIBRARY: /usr/lib/x86_64-linux-gnu/libOpenGL.so
    OPENGL_glx_LIBRARY: /usr/lib/x86_64-linux-gnu/libGLX.so

  OpenGL_GL_PREFERENCE has not been set to "GLVND" or "LEGACY", so for
  compatibility with CMake 3.10 and below the legacy GL library will be used.
Call Stack (most recent call first):
  CMakeLists.txt:22 (find_package)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Configuring done
-- Generating done
-- Build files have been written to: /home/rakuli/3dTracking/3DObjectTracking/SRT3D

（5）make

~/3dTracking/3DObjectTracking/SRT3D/examples$ make
[ 51%] Built target srt3d
[ 62%] Built target evaluate_rbot_dataset
[ 72%] Built target parameter_study_rbot_dataset
[ 75%] Building CXX object examples/CMakeFiles/evaluate_opt_dataset.dir/evaluate_opt_dataset.cpp.o
[ 79%] Linking CXX executable evaluate_opt_dataset
[ 82%] Built target evaluate_opt_dataset
[ 86%] Building CXX object examples/CMakeFiles/parameter_study_opt_dataset.dir/parameter_study_opt_dataset.cpp.o
[ 89%] Linking CXX executable parameter_study_opt_dataset
[ 93%] Built target parameter_study_opt_dataset
[100%] Built target run_on_recorded_sequence

（6）将examples中与opt相关的6个文件复制到src文件夹中

weixin_54470372
关注
  • 0
    点赞
  • 踩
  • 10
    收藏
    觉得还不错? 一键收藏
  • 4
    评论
专栏目录
传感器融合-SFND_3D_Object_Tracking环境配置
qq_33287871的博客
06-11 832
1.环境: cmake >= 2.8 All OSes: click here for installation instructions make >= 4.1 (Linux, Mac), 3.81 (Windows) Linux: make is installed by default on most Linux distros Mac: install Xcode command line tools to get make Windows: Click here for install
RBOT: Region-based Object Tracking
moonlightpeng的博客
05-01 598
githubhttps://github.com/henningtjaden/RBOT Related Papers A Region-based Gauss-Newton Approach to Real-Time Monocular Multiple Object TrackingH. Tjaden, U. Schwanecke, E. Schömer, D. Cremers, TPAMI '18 Real-Time Monocular Pose Estimation o...
awesome-3D-object-detection:3D物体检测
04-15
作者：Tom Hardy 公众号： 公众号运营者和嘉宾介绍：运营者来自国内一线大厂的算法工程师，深研3D视觉、vSLAM、计算机视觉、点云处理、深度学习、自动驾驶、图像处理、三维重建等领域，特邀嘉宾包括国内外知名高校的博士硕士，旷视、商汤、百度、阿里等就职的算法大佬，欢迎一起交流学习！ 主要针对3D object相关算法进行了汇总，分为基于RGB图像、RGB-D数据、立体视觉、点云、融合等方式，欢迎补充~ 一、基于点云的三维目标检测算法 STD: Sparse-to-Dense 3D Object Detector for Point Cloud Fast Point R-CNN StarNet: Targeted Computation for Object Detection in Point Clouds Class-balanced Grouping and Sampling fo
ubuntu20.04+Vtk 7.1.1+pcl 安装过程中出现的问题
m0_48919875的博客
06-24 2531
pcl安装过程中遇到的一些问题
cmakelist 报 legacy GL 与OpenGL and GLX 警告问题
u010815558的博客
05-17 7628
安装Nvidia 驱动的时候勾选 openGl选项造成的。 cmake警告如下 CMake Warning (dev) at /home/××/Programs/clion-2019.1/bin/cmake/linux/share/cmake-3.13/Modules/FindOpenGL.cmake:270 (message): Policy CMP0072 is not set: FindOp...
点云 3D 目标检测 - CenterPoint：Center-based 3D Object Detection and Tracking（CVPR 2021）
77wpa的博客
03-08 5159
三维目标通常表示为点云中的三维长方体。这种表示模拟了经过充分研究的基于图像的2D边界框检测，但存在额外的挑战。三维世界中的目标不遵循任何特定的方向，基于框的检测器很难枚举所有方向或将轴对齐的边界框拟合到旋转的目标。在本文中，我们建议将3D目标表示、检测和跟踪为点。我们的框架CenterPoint首先使用关键点检测器检测目标的中心，然后回归到其他属性，包括3D大小、3D方向和速度。在第二阶段，它使用目标上的其他点特征来细化这些估计。在CenterPoint中，3D目标跟踪简化为贪婪的最近点匹配。
【复现笔记】Iterative Corresponding Geometry
weixin_54470372的博客
10-30 1207
Iterative Corresponding Geometry: Fusing Region and Depth for Highly Efficient 3D Tracking of Textureless Objects CVPR2022
Multimodal Sparse Representation-Based Classification for Lung Needle Biopsy Images
02-21
In this study, a novel method, multimodal sparse representation-based classification (mSRC), is proposed for classifying lung needle biopsy images. In the data acquisition procedure of our method, ...
sparse-checkout-example：git sparse-checkout功能的示例仓库
02-25
git sparse-checkout命令您可以使用以下团队之一运行bootstrap.sh命令： android ， identity或browser 。 每个人都将启动git sparse-checkout set命令，以仅显示这些团队所需的目录。归因这些项目中的代码仅作为...
3D-sparse-octree.zip
09-17
而在这个过程中，3D-sparse-octree（3D稀疏八叉树）数据结构是其中的核心技术之一，它为高效地存储、管理和操作3D模型提供了可能。 首先，理解3D八叉树的概念至关重要。八叉树是一种分层的数据结构，每个节点有八...
sparse-segmentation:RISS 2018-稀疏LIDAR点云的分割
04-29
一个基于C ++的点云地面注释器，基于：3D点云的快速分割：LIDAR数据的范式作者：Dimitris Zermas Izzat Izzat和Nikolas Papanikolopoulos @inproceedings{7989591, author={D. Zermas and I. Izzat and N. ...
ubuntu20.04 安装RealSenseD435驱动
Baby_of_breath的博客
07-10 5252
在github中，选择默认的master分支。下载后，解压缩到home目录（个人习惯，可放到别处）。 3. 安装权限脚本 （1）进入到librealsense目录(笔者是放到/home下) （2）输入以下命令 4.2 新建build文件夹，并进入 4.3 编译 （1）第一步： 该步运行成功标志： 运行失败标志： （2）第二步： 原因：不能从github上，下载curl make指令遇到报错1时，第一步的cmake …指令也会失败。参考该作者，他下载的是curl-7.61.0。我自己尝试该版本，
3dobjectTracking配置ICG指导意见
liyunfengdi的专栏
12-09 364
3dobjectTracking配置ICG指导意见
ubuntu下PCL库编译安装
z1314520cz
01-06 6727
从PCL github仓库中下载最新的release, 解压并进入目录中, 创建build目录并进入, 在该目录中进行make编译安装 我的GPU是a卡, 用不了cuda框架, 在build目录下执行以下命令完成编译安装: cmake .. make -j4 sudo make install 第一行命令是很关键的, 在第一行命令运行通过后, 下面两行命令就顺利地跑完了。所以在编译安装中最...
关于SLAM十四讲第9讲的编译过程报错问题！！！（/usr/bin/ld: CMakeFiles/bundle_adjustment_g2o.dir/bundle_adjustment_g2o.cp）
m0_68963624的博客
03-26 1349
后来想到是不是第一次libsuitesparse库没卸载干净，导致第二次sudo apt install libsuitesparse库的时候与第一次残余文件冲突，导致g2o编译时链接到的是第一次的残余文件（PS：这里是我的一个猜测，至于是不是希望有大佬解答！这里提供用来删除软件（库）等残余文件的命令，它可以在删除软件本身的同时，删除该软件包的残余设置文件。另外，查了好多别人的博客记录，好像没有其他人出现我这个错误！这其中肯定会报错的，但是这里就能通过报错的内容把。，只留下不报错的自然就能编译通过！
CMake 常见问题及解决办法
最新发布
一歲抬頭的博客
08-01 1万+
本文介绍了一些CMake常见问题及解决办法，主要涉及到Boost库、安装命令、编译器、生成器、缓存文件、路径转义和版本更新等方面。希望本文能够帮助大家更好地使用CMake构建自己的项目。
cmake对编译器版本和库依赖的问题
fpcc的专栏
10-21 1万+
cmake对编译器版本和库依赖的问题 一、介绍 最近在编译Wasm的几个相关工具时，其使用的是cmake工具，一般来说，开源的软件的编译文档会写的相当清楚，而且开源软件为了让客户体验方便，一般编译的步骤和依赖的相关工具和库，都会尽量整合到一起，防止因为无谓的原因导致编译问题影响软件的整体下载应用。 在编译相关的工具过程中，确实发现很多都基本可以做到两三到命令即可完成编译，也不需要下载过多的依赖。但...
Ubuntu 20.04.05安装PCL-1.12.0
zhiTjun的博客
11-27 3179
Ubuntu 20.04.05安装QT+VTK+PCL
sparse r-cnn: end-to-end object detection with learnable proposals
03-16
Sparse R-CNN是一种端到端的物体检测方法，其中使用可学习的提议来检测目标。与传统的R-CNN方法不同，Sparse R-CNN使用了稀疏的特征图来提高检测效率。该方法在多个数据集上进行了测试，并取得了很好的性能。

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 4
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值