OAK-D-PRO相机运行orb_slam3双目+IMU

1.参照官方教程安装depthai_ros；1.1. ROS1 noetic + depthai_ros教程 — DepthAI Docs 0.3.0.0 documentation

2.安装好depthai_ros之后测试下面指令：

roslaunch depthai_examples stereo_inertial_node.launch  depth_aligned:=false

如下图所示：

​

3.接下来下载编译OPR_SLAM3,参照安装ORB-SLAM3教程-CSDN博客

4.查看并修改相机参数，编写相机的内参文件 oak-d-pro.yaml

4.1 IMU和双目相机标定参考1.3. ubuntu 18.04 OAK-D kalibr 双目+imu联合标定 — DepthAI Docs 0.3.0.0 documentation

4.2 编写相机的内参文件 oak-d-pro.yaml，下面是我的配置文件，可作参考。

%YAML:1.0
 #自己用的时候将汉字备注全部删除，不然可能会报错
#--------------------------------------------------------------------------------------------
# System config
#--------------------------------------------------------------------------------------------
 
# When the variables are commented, the system doesn't load a previous session or not store the current one
# If the LoadFile doesn't exist, the system give a message and create a new Atlas from scratch
# System.LoadAtlasFromFile: "MapEuRoCMH_StereoImu"
 
# The store file is created from the current session, if a file with the same name exists it is deleted
# System.SaveAtlasToFile: "MapEuRoCMH_StereoImu"
 
#--------------------------------------------------------------------------------------------
# Camera Parameters. Adjust them!
#--------------------------------------------------------------------------------------------
File.version: "1.0"
 
Camera.type: "PinHole"
 
# Camera calibration and distortion parameters (OpenCV)
# left
Camera1.fx: 805.9707277371266   #根据实际填写

Camera1.fy: 804.9165005674341    #根据实际填写
Camera1.cx: 666.816075048802    #根据实际填写
Camera1.cy: 365.13825670241545  #根据实际填写
 
Camera1.k1: 0.018348427806329223  #根据实际填写
Camera1.k2: -0.07242223628816746 #根据实际填写
Camera1.p1: 0.0014193412480854573 #根据实际填写
Camera1.p2: 0.002458344231723593 #根据实际填写
 
#right
Camera2.fx: 804.7413191715098  #根据实际填写
Camera2.fy: 802.6525823682881  #根据实际填写
Camera2.cx: 669.3381686190725  #根据实际填写
Camera2.cy: 363.5067790321098  #根据实际填写
 
Camera2.k1: -0.003977004249998143 #根据实际填写
Camera2.k2: -0.036977907124922015 #根据实际填写
Camera2.p1: 0.0015061674991008577 #根据实际填写
Camera2.p2: 0.0014536300602192025 #根据实际填写
 
Camera.width: 1280 #根据实际填写
Camera.height: 720 #根据实际填写
 
# Camera frames per second
Camera.fps: 30
 
# Color order of the images (0: BGR, 1: RGB. It is ignored if images are grayscale)
Camera.RGB: 1
 
# Close/Far threshold. Baseline times.
Stereo.ThDepth: 10000.0
Stereo.T_c1_c2: !!opencv-matrix   #cam2 to cam1 #根据实际填写
  rows: 4
  cols: 4
  dt: f
  data: [0.9999998191890077, -0.0005994506057517135, 4.775891176709104e-05, -0.07383358902408807,
        0.000599449606536203, 0.9999998201109549, 2.093364263618778e-05, 0.0002542064670240248,
        -4.777145186054509e-05, -2.09050097902878e-05, 0.9999999986404297, -0.0003246086293178939,
        0.0, 0.0, 0.0, 1.0]
 
 
# Transformation from camera 0 to body-frame (imu) cam0 to imu #根据实际填写
IMU.T_b_c1: !!opencv-matrix
  rows: 4
  cols: 4
  dt: f
  # data: [-0.0234653, 0.00342746, -0.999719, -0.172357,
  #        0.999722, 0.0021686,-0.0234579, -0.0610075,
  #        0.00208759, -0.999992, -0.00347748, -0.0814507,
  #        0.0, 0.0, 0.0, 1.0]
  data: [-0.01018823, -0.99987712, -0.0119141 ,  0.03462867,
          -0.99945173,  0.01055782, -0.03138098,  0.07278766,
           0.03150292,  0.01158786, -0.99943649, -0.01794634,
           0,          0,         0  ,        1       ]
 
# IMU noise
# IMU.NoiseGyro: 0.00010426654551109309
# IMU.NoiseAcc: 0.0002276846771973415
# IMU.GyroWalk: 3.5636559575381104e-06
# IMU.AccWalk: 6.706087584689249e-06
# IMU.Frequency: 200.0
 

IMU.NoiseGyro: 5.3507436729098995e-03 #根据实际填写
IMU.NoiseAcc: 2.6149296082711892e-02  #根据实际填写
IMU.GyroWalk: 2.8903125733944023e-05  #根据实际填写
IMU.AccWalk: 6.2858306116533641e-04   #根据实际填写
IMU.Frequency: 200.0
 
 
#--------------------------------------------------------------------------------------------
# ORB Parameters
#--------------------------------------------------------------------------------------------
 
# ORB Extractor: Number of features per image
ORBextractor.nFeatures: 1200
 
# ORB Extractor: Scale factor between levels in the scale pyramid     
ORBextractor.scaleFactor: 1.2
 
# ORB Extractor: Number of levels in the scale pyramid    
ORBextractor.nLevels: 8
 
# ORB Extractor: Fast threshold
# Image is divided in a grid. At each cell FAST are extracted imposing a minimum response.
# Firstly we impose iniThFAST. If no corners are detected we impose a lower value minThFAST
# You can lower these values if your images have low contrast            
ORBextractor.iniThFAST: 20
ORBextractor.minThFAST: 7
 
#--------------------------------------------------------------------------------------------
# Viewer Parameters
#--------------------------------------------------------------------------------------------
Viewer.KeyFrameSize: 0.05
Viewer.KeyFrameLineWidth: 1.0
Viewer.GraphLineWidth: 0.9
Viewer.PointSize: 2.0
Viewer.CameraSize: 0.08
Viewer.CameraLineWidth: 3.0
Viewer.ViewpointX: 0.0
Viewer.ViewpointY: -1.7
Viewer.ViewpointZ: 0.8
Viewer.ViewpointF: -500.0
Viewer.imageViewScale: 1.0

 5.修改话题

打开～/ORB_SLAM3/Examples_old/ROS/ORB_SLAM3/src/ros_stereo_inertial.cc，话题修改为下图所示：

​

 保存后重新编译：

./build_ros.sh

 6.启动

启动oak-d-pro/poe双目相机指令

source ./devel/setup.bash

roslaunch depthai_examples stereo_inertial_node.launch enableRviz:=false depth_aligned:=false

在orb_slam3工作区下运行节点

rosrun ORB_SLAM3 Stereo_Inertial /home/lzd/ORB_SLAM3/Vocabulary/ORBvoc.txt /home/lzd/ORB_SLAM3/Examples_old/Stereo-Inertial/oak-d-pro.yaml false

 注意：ORBvoc.txt和oak-d-pro.yaml两个文件的路径改为你自己的实际路径