ORB-SLAM2复现
1 环境准备
Ubuntu18.04
1.1 Pangolin
安装依赖
sudo apt -get install libglew-dev
sudo apt -get install libboost-dev libboost-thread-dev libboost-filesystem-dev
sudo apt -get install libpython2.7-dev
选择0.5版本的Pangolin进行源码编译安装。
# 最后的参数使得git的包名为Pangolin
git clone https://github.com/stevenlovegrove/Pangolin/tree/v0.5 Pangolin
cd Pangolin
mkdir build && cd build
cmake ..
make -j4
sudo make install # 安装
1.2 OpenCV
OpenCV，这里选择了3.4.16
- 安装依赖
sudo apt install build-essential cmake git pkg-config libgtk-3-dev \
libavcodec-dev libavformat-dev libswscale-dev libv4l-dev \
libxvidcore-dev libx264-dev libjpeg-dev libpng-dev libtiff-dev \
gfortran openexr libatlas-base-dev python3-dev python3-numpy \
libtbb2 libtbb-dev libdc1394-22-dev libopenexr-dev \
libgstreamer-plugins-base1.0-dev libgstreamer1.0-dev # 根据需要安装
- 源码编译
cd opencv-3.4.16
mkdir build && cd build
cmake ..
make -j4 # 可能会有点慢
sudo make install
1.3 Eigen3
sudo apt install libeigen3-dev # 直接命令行安装即可
1.4 DBoW2 and g2o
作者进行了修改，直接在ORB源码编译时进行安装
2 C++编译
2.1 下载源码
git clone https://github.com/raulmur/ORB_SLAM2.git ORB_SLAM2
cd ORB_SLAM2
chmod +x build.sh
gedit build.sh
2.2 build.sh
# 1.编译DBoW2
cd Thirdparty/DBoW2
mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Release
make -j
# 2.编译g2o
cd ../../g2o
mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Release
make -j
# 3 解压ORB字典
cd Vocabulary
tar -xf ORBvoc.txt.tar.gz
cd ..
# 4.编译ORB源码
mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Release
make -j
2.3 报错
./build.sh # 虚拟机性能不行的话，第一次可以一步一步按上面命令单独执行
# 把下面两句放在CMakeLists.txt来减少不必要的警告
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-deprecated-declarations")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-deprecated")
① 报错1
报错：ORB/ORB_SLAM2-master/src/System.cc:134:17: error: ‘usleep’ was not declared in this scope usleep(1000); ^~~~~~
即使用了未定义的引用，也就是缺少相应的头文件，很明显指System.cc中缺失，然后就会发现很多文件都缺失了这个头文件，都补上即可。
#include <unistd.h> // 在上面文件加上头文件即可消除报错
System.cc
Tracking.cc
LocalMapping.cc
LoopClosing.cc
Viewer.cc
mono_euroc.cc
mono_kitti.cc
mono_tum.cc
stereo_kitti.cc
stereo_euroc.cc
rgbd_tum.cc
②报错2
出现这种问题，就是pangolin的版本不对，换成0.5就ok，否则在make时候会找不到/usr/bin/ld: cannot find -leigen3
CMake Warning (dev) at CMakeLists.txt:103 (add_executable):
Policy CMP0028 is not set: Double colon in target name means ALIAS or
IMPORTED target. Run "cmake --help-policy CMP0028" for policy details.
Use the cmake_policy command to set the policy and suppress this warning.
Target "stereo_kitti" links to target "Eigen3::Eigen" but the target was
not found. Perhaps a find_package() call is missing for an IMPORTED
target, or an ALIAS target is missing?
This warning is for project developers. Use -Wno-dev to suppress it.
CMake Warning (dev) at CMakeLists.txt:107 (add_executable):
Policy CMP0028 is not set: Double colon in target name means ALIAS or
IMPORTED target. Run "cmake --help-policy CMP0028" for policy details.
Use the cmake_policy command to set the policy and suppress this warning.
Target "stereo_euroc" links to target "Eigen3::Eigen" but the target was
not found. Perhaps a find_package() call is missing for an IMPORTED
target, or an ALIAS target is missing?
This warning is for project developers. Use -Wno-dev to suppress it.
CMake Warning (dev) at CMakeLists.txt:107 (add_executable):
Policy CMP0028 is not set: Double colon in target name means ALIAS or
IMPORTED target. Run "cmake --help-policy CMP0028" for policy details.
Use the cmake_policy command to set the policy and suppress this warning.
Target "stereo_euroc" links to target "Eigen3::Eigen" but the target was
not found. Perhaps a find_package() call is missing for an IMPORTED
target, or an ALIAS target is missing?
This warning is for project developers. Use -Wno-dev to suppress it.
CMake Warning (dev) at CMakeLists.txt:107 (add_executable):
Policy CMP0028 is not set: Double colon in target name means ALIAS or
IMPORTED target. Run "cmake --help-policy CMP0028" for policy details.
Use the cmake_policy command to set the policy and suppress this warning.
Target "stereo_euroc" links to target "Eigen3::Eigen" but the target was
not found. Perhaps a find_package() call is missing for an IMPORTED
target, or an ALIAS target is missing?
This warning is for project developers. Use -Wno-dev to suppress it.
2.4 配置文件
以KITTI00-02.yaml为例
Camera.fx: 718.856 # 内参K
Camera.fy: 718.856
Camera.cx: 607.1928
Camera.cy: 185.2157
Camera.k1: 0.0 # 畸变参数
Camera.k2: 0.0
Camera.p1: 0.0
Camera.p2: 0.0
Camera.width: 1241 # 图像宽高
Camera.height: 376
Camera.fps: 10.0 # 相机帧率
Camera.bf: 386.1448 # 双目基线b
# Color order of the images (0: BGR, 1: RGB. It is ignored if images are grayscale)
Camera.RGB: 1
# Close/Far threshold. Baseline times. 远近点阈值
ThDepth: 35
#--------------------------------------------------------------------------------------------
# ORB Parameters 特征提取参数
#--------------------------------------------------------------------------------------------
ORBextractor.nFeatures: 2000
ORBextractor.scaleFactor: 1.2
ORBextractor.nLevels: 8
ORBextractor.iniThFAST: 20
ORBextractor.minThFAST: 7
#--------------------------------------------------------------------------------------------
# Viewer Parameters 可视化参数
#--------------------------------------------------------------------------------------------
Viewer.KeyFrameSize: 0.6
Viewer.KeyFrameLineWidth: 2
Viewer.GraphLineWidth: 1
Viewer.PointSize:2
Viewer.CameraSize: 0.7
Viewer.CameraLineWidth: 3
Viewer.ViewpointX: 0
Viewer.ViewpointY: -100
Viewer.ViewpointZ: -0.1
Viewer.ViewpointF: 2000
3 C++下运行
3.1 Monocular
3.1.1 KITTI Dataset
mono_kitti.cc
// arg0：可执行文件 arg1：字典路径 arg2：配置文件yaml arg3：数据路径
if(argc != 4)
{
cerr << endl << "Usage: ./mono_kitti path_to_vocabulary path_to_settings path_to_sequence" << endl;
return 1;
}
...
SLAM.SaveKeyFrameTrajectoryTUM("KeyFrameTrajectory.txt"); // 保存的关键帧轨迹 TMU数据格式
- 执行
cd ORB_SLAM2
./Examples/Monocular/mono_kitti Vocabulary/ORBvoc.txt Examples/Monocular/KITTIX.yaml xxx # 这里写数据路径 敲的时候最好利用tab补全，避免打错
# /mnt/hgfs/dataset/KITTI/rgb/04/ 这里跑了KITTI04数据集，数据结果
3.1.2 TUM Dataset
mono_tum.cc
{
// arg0:可执行文件 arg1:字典路径 arg2:配置文件 arg3:图像路径
cerr << endl << "Usage: ./mono_tum path_to_vocabulary path_to_settings path_to_sequence" << endl;
return 1;
}
...
SLAM.SaveKeyFrameTrajectoryTUM("KeyFrameTrajectory.txt");
- 执行
cd ORB_SLAM2
./Examples/Monocular/mono_tum Vocabulary/ORBvoc.txt Examples/Monocular/TUM3.yaml /mnt/hgfs/TUM/rgbd_dataset_freiburg3_long_office_household
# 注意这里的配置文件要和数据文件对应！
3.1.3 EuRoC Dataset
数据地址，这里提供了ROS bag和ASL Dataset Format
mono_euroc.cc
if(argc != 5)
{
// arg0:可执行文件 arg1:字典路径 arg2：相机配置文件ymal arg3：图像路径 arg4：时间戳路径
cerr << endl << "Usage: ./mono_tum path_to_vocabulary path_to_settings path_to_image_folder path_to_times_file" << endl;
return 1;
}
...
SLAM.SaveKeyFrameTrajectoryTUM("KeyFrameTrajectory.txt");
- 执行
cd ORB_SLAM2
./Examples/Monocular/mono_euroc Vocabulary/ORBvoc.txt Examples/Monocular/EuRoC.yaml PATH_TO_SEQUENCE_FOLDER/mav0/cam0/data # 图像路径
Examples/Monocular/EuRoC_TimeStamps/SEQUENCE.txt # 时间戳路径
3.2 Stereo
3.2.1 KITTI Dataset
stereo_kitti.cc
// arg0：可执行文件 arg1：字典路径 arg2：配置文件yaml arg3：数据路径
if(argc != 4)
{
cerr << endl << "Usage: ./stereo_kitti path_to_vocabulary path_to_settings path_to_sequence" << endl;
return 1;
}
...
SLAM.SaveKeyFrameTrajectoryTUM("KeyFrameTrajectory.txt"); // 保存的关键帧轨迹 TMU数据格式
- 执行
cd ORB_SLAM2
./Examples/Stereo/stereo_kitti Vocabulary/ORBvoc.txt Examples/Stereo/KITTIX.yaml PATH_TO_DATASET_FOLDER/dataset/sequences/SEQUENCE_NUMBER
KITTI00双目，如果这里用的还是单目的配置文件，就会启动失败！两个配置文件是有差异的，比如ThDepth: 35
3.2.2 EuRoC Dataset
stereo_euroc.cc
if(argc != 5)
{
// arg0:可执行文件 arg1:字典路径 arg2：相机配置文件ymal arg3：左目图像路径
// arg4：右目图像路径 arg5：时间戳路径
cerr << endl << "Usage: ./stereo_euroc path_to_vocabulary path_to_settings path_to_left_folder path_to_right_folder path_to_times_file" << endl;
return 1;
}
...
SLAM.SaveKeyFrameTrajectoryTUM("KeyFrameTrajectory.txt");
- 执行
cd ORB_SLAM2
./Examples/Stereo/stereo_euroc Vocabulary/ORBvoc.txt Examples/Stereo/EuRoC.yaml /mnt/hgfs/euroc/mav0/cam0/data /mnt/hgfs/euroc/mav0/cam1/data Examples/Stereo/EuRoC_TimeStamps/MH05.txt
MH05数据集
3.3 RGB-D(TUM)
3.3.1 关联数据
需要通过该网站中的 associate.py 来对彩色RGB图片和深度depth图进行关联。
Examples/RGB-D/associations/ 目录中已经提供了一些已经关联好的数据数列。可以通过以下命令生成自己的数据序列：
python associate.py PATH_TO_SEQUENCE/rgb.txt PATH_TO_SEQUENCE/depth.txt > associations.txt
以Kinect为例，其原理是以非同步的方式提供颜色和深度图像。这意味着彩色图像的时间戳与深度图像的时间戳不会相交。因此，需要找到一种将彩色图像与深度图像相关联的方法。
为了这个目的，使用 associate.py 进行关联，其原理是读取rgb.txt和depth.txt的时间戳信息，然后把最接近的匹配到一起。
3.3.2 测试
rgbd_tum.cc
int main(int argc, char **argv)
{
if(argc != 5)
{
cerr << endl << "Usage: ./rgbd_tum path_to_vocabulary path_to_settings path_to_sequence path_to_association" << endl;
return 1;
}
}
- 执行
./Examples/RGB-D/rgbd_tum Vocabulary/ORBvoc.txt Examples/RGB-D/TUM3.yaml PAHT-to/TUM/rgbd_dataset_freiburg3_long_office_household ./Examples/RGB-D/accelerometer_long_office_household.txt
# 如果报错了，一般是关联数据的txt文件有问题
4 运行报错
4.1 报错1：Failed to load image
报错：不能加载数据。原因要么是数据路径有问题，或者就是配置有问题，就是双目的ymal和使用的rgb数据等
4.2 无法保存轨迹
用ORB-SLAM2运行KITTI数据集时，最后窗口卡住，无法保存轨迹文件。
// 注释system.cc文件中这两行 319-320行
if (mpViewer)
pangolin::BindToContext("ORB-SLAM2: Map Viewer");
5 ROS编译
5.1 编译
./build_ros.sh
5.2 报错
① 报错1
rospack found package "ORB_SLAM2" at "", but the currentdirectory is "/home/p/ORB_SLAM2-master/Examples/ROS/ORB_SLAM2 报错的意思是
rospack find ORB_SLAM2 失败，找不到这个功能包，解决办法就是把功能包的路径加入到${ROS_PACKAGE_PATH}，然后再次编译即可成功（如果还是报同样的错，那就是路径不对，比如我这里一直报错因为home前面/忘打了）。
gedit ~/.bashrc # 然后在最后加上下面这句
export ROS_PACKAGE_PATH=${ROS_PACKAGE_PATH}:/home/p/ORB_SLAM2/Examples/ROS
source ~/.bashrc # 刷新
# 验证修改
echo ${ROS_PACKAGE_PATH} # 查看路径
p@p:~/ORB_SLAM2-master/Examples/ROS/ORB_SLAM2$ rospack list # 直接查找
ORB_SLAM2 /home/p/ORB_SLAM2-master/Examples/ROS/ORB_SLAM2
....
② 报错2
Failed to invoke rospack to get compile flags for package 'ORB_SLAM2'. Look above for errors from rospack itself. Aborting. Please fix the broken dependency!
$ sudo rosdep fix-permissions
$ rosdep update
③ 报错3
undefined reference to symbol '_ZN5boost6system15system_categoryEv'
set(LIBS
${OpenCV_LIBS}
${EIGEN3_LIBS}
${Pangolin_LIBRARIES}
${PROJECT_SOURCE_DIR}/../../../Thirdparty/DBoW2/lib/libDBoW2.so
${PROJECT_SOURCE_DIR}/../../../Thirdparty/g2o/lib/libg2o.so
${PROJECT_SOURCE_DIR}/../../../lib/libORB_SLAM2.so
-lboost_system # 把这个加上 ROS下的那个CMakeLists.txt
)
④ 报错4
ViewerAR.cc:233:9: error: ‘usleep’ was not declared in this scope usleep(mT*1000);
#include <unistd.h> // 在上面文件加上头文件即可消除报错
6 ROS运行测试
6.1 测试单目节点
记得改一下配置文件，没有图像就是话题名错误，改下程序中的订阅话题（或在命令行中更改）。运行单目增强流程和这里差不多。
rosrun ORB_SLAM2 Mono PATH_TO_VOCABULARY PATH_TO_SETTINGS_FILE _rgb:=/camera/rgb/image_raw
6.2 测试RGB-D
利用Astra pro相机进行测试
roslaunch astra_camera astrapro.launch # 要开始相机节点
rostopic list # 查看相关话题名，方便订阅图像数据
rosrun ORB_SLAM2 RGBD Vocabulary/ORBvoc.txt Examples/ROS/ORB_SLAM2/Asus.yaml _rgb:=/camera/rgb/image_raw _depth:=/camera/depth_registered/image_raw
6.3 测试双目数据集
rosrun ORB_SLAM2 Stereo PATH_TO_VOCABULARY PATH_TO_SETTINGS_FILE ONLINE_RECTIFICATION
rosrun ORB_SLAM2 Stereo Vocabulary/ORBvoc.txt Examples/Stereo/EuRoC.yaml true
rosbag play --pause V1_01_easy.bag /cam0/image_raw:=/camera/left/image_raw /cam1/image_raw:=/camera/right/image_raw # 没有双目相机，就利用ros数据
7 运行自己的数据集
以KITTI为例，需要准备配置文件(相机内参和畸变系数)和时间戳文件。
假设有一段视频数据，我们利用python程序把它拆分为图像数据，并生成时间戳文件。接着按上面的方法运行即可，但是发现很多时候会跟丢，例如拐弯处。
# -*- coding: utf-8 -*-
import cv2
import os
from datetime import datetime
def split_video_to_frames(video_path, output_folder):
cap = cv2.VideoCapture(video_path)
frame_count = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
fps = int(cap.get(cv2.CAP_PROP_FPS))
if not os.path.exists(output_folder):
os.makedirs(output_folder)
timestamps = []
for i in range(frame_count):
ret, frame = cap.read()
if not ret:
break
timestamp = "{:.6e}".format(i / fps)
timestamps.append(timestamp)
frame_filename = os.path.join(output_folder, '{:06d}.jpg'.format(i))
cv2.imwrite(frame_filename, frame)
cap.release()
return timestamps
# 设置视频路径和输出文件夹
video_path = 'video.mp4'
output_folder = 'data'
# 拆分视频，并记录时间戳
timestamps = split_video_to_frames(video_path, output_folder)
# 保存时间戳到文件
timestamps_file = os.path.join(output_folder, 'times.txt')
with open(timestamps_file, 'w') as f:
f.write('\n'.join(timestamps))
4639
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
