Kalibr tutorials

Kalibr installation tutorial

I was confused about installing Kalibr, but there is no even one hint in README.md. I just put them in the catkin_ws, in which so many ROS packages are also there. Unsuccessfully, it can't be compiled one by one package by the command catkin_make -DCATKIN_WHITELIST_PACKAGE="PACKAGE_NAME". It means a good choice is to build another ROS workspace in case of rebuilding others in the same workspace.

---

Resiquite:

sudo apt-get install python-setuptools python-rosinstall ipython libeigen3-dev libboost-all-dev doxygen libopencv-dev ros-kinetic-vision-opencv ros-kinetic-image-transport-plugins ros-kinetic-cmake-modules python-software-properties software-properties-common libpoco-dev python-matplotlib python-scipy python-git python-pip ipython libtbb-dev libblas-dev liblapack-dev python-catkin-tools libv4l-dev

sudo pip install python-igraph --upgrade

---

Warning: If having done catkin_make at first then must run the following command.

catkin clean -bdy

---

cd ~
mkdir -p kalibr_ws/src
cd ~/kalibr_workspace 
source /opt/ros/indigo/setup.bash
catkin init 
catkin config --extend /opt/ros/kinetic
catkin config --merge-devel # Necessary for catkin_tools >= 0.4. catkin config --cmake-args -DCMAKE_BUILD_TYPE=Release
cd ~/kalibr_ws/src
git clone https://github.com/ethz-asl/kalibr.git
cd ..
catkin build -DCMAKE_BUILD_TYPE=Release -j4

Output seems like this:

Finished  <<< kalibr                                        [ 16.1 seconds ]                                                                                           
[build] Summary: All 37 packages succeeded!                                                                                                                            
[build]   Ignored:   None.                                                                                                                                             
[build]   Warnings:  21 packages succeeded with warnings.                                                                                                              
[build]   Abandoned: None.                                                                                                                                             
[build]   Failed:    None.                                                                                                                                             
[build] Runtime: 14 minutes and 53.4 seconds total.                                                                                                                    
[build] Note: Workspace packages have changed, please re-source setup files to use them.

source ~/kalibr_ws/devel/setup.bash

Update:

---

Traceback (most recent call last):
  File "../python/kalibr_calibrate_cameras", line 6, in <module>
    import sm
ImportError: No module named sm

Solution:

sudo pip install sm

then rebuild kalibr.

---

References:
[1] kalibr教程
[2] Installing and Configuring Your ROS Environment
[3] ethz-asl/kalibr
[4] catkin_make vs catkin build
[5] https://github.com/ethz-asl/kalibr/wiki/installation
[6] 完整版用kalibr标定 camera imu

Multiple camera calibration

---

roslaunch realsense2_camera rs_camera.launch
rosrun topic_tools throttle messages /camera/color/image_raw 4.0 /color
rosbag record -O rs_cam_hz4 /color

Which distortiong model should be choose for Realsense D435i? From all I know, a factory calibration setup of D435i looks like: (You can /usr/local/bin/rs-sensor-control, type 0, 1, 2, 91 etc to see)

Principal Point         : 322.424, 237.813
Focal Length            : 617.521, 617.576
Distortion Model        : Brown Conrady
Distortion Coefficients : [0,0,0,0,0]

And according to the dorodnic, of course a equidistant distortion model could be used. (But r1 & r2 are needed in realsense comfig in vins. So the best distortion model must be radial-tangential (radtan))

Yes, these are supposed to be zero for the D400. We consider adding coefficient estimation to the RGB calibration to reduce the distortion (by about 1 pixel at extremes), but at the moment projection without coefficients is the most accurate you can do (we are not calibrating and then ignoring the coefficients, we estimate fx, fy, ppx and ppy without them)

cd ~/kalibr_ws/src/kalibr/aslam_offline_calibration/kalibr/data
../python/kalibr_calibrate_cameras --target april_6x6_50x50cm.yaml --bag rs_cam_hz4.bag --models pinhole-equi --topics /color

Note that in the bag file there are up to 800 images, but it only 39. Maybe that's enough for calibration?

Output:

Calibration complete.

[ WARN] [1556719991.003758]: Removed 26 outlier corners.

Processed 826 images with 39 images used
Camera-system parameters:
    cam0 (/color):
     type: <class 'aslam_cv.libaslam_cv_python.EquidistantDistortedPinholeCameraGeometry'>
     distortion: [  0.3044413    2.04741574 -11.06112629  18.6743852 ] +- [ 0.0320288   0.46759766  2.76374537  5.41971393]
     projection: [ 604.9671891   602.10506316  325.8395051   238.35406753] +- [ 10.62286295  10.41921913   1.68531874   1.43868064]
     reprojection error: [-0.000000, -0.000000] +- [0.153693, 0.138547]

Results written to file: camchain-rs_cam_hz4.yaml
  Detailed results written to file: results-cam-rs_cam_hz4.txt

Result:

camchain-rs_cam_hz4.yaml

cam0:
  cam_overlaps: []
  camera_model: pinhole
  distortion_coeffs: [0.3044413037380324, 2.0474157424478348, -11.061126286843251,
    18.67438520203368]
  distortion_model: equidistant
  intrinsics: [604.9671890973748, 602.1050631617551, 325.83950509989114, 238.35406753467785]
  resolution: [640, 480]
  rostopic: /color

Compared to the default settings, assumes our result is accurate. The reprojection error seems like good too.

Reference:
[1] Multiple camera calibration
[2] [相机标定]RealSense D435i相机标定
[3] rs2_intrinsics coeffs[] all 0 by default #1430
[4] Camera models

IMU calibration

---

imu_utils from HKUST

Protecting from error:

CMake Warning at /opt/ros/kinetic/share/catkin/cmake/catkinConfig.cmake:76 (find_package):
Could not find a package configuration file provided by "code_utils" with
any of the following names:
code_utilsConfig.cmake
code_utils-config.cmake
Add the installation prefix of "code_utils" to CMAKE_PREFIX_PATH or set
"code_utils_DIR" to a directory containing one of the above files. If
"code_utils" provides a separate development package or SDK, be sure it has
been installed.

Put code_utils in the workspace, catkin_make first.

Then do the same for imu_utils.

Result (BMI055 is the IMU D435i is using):

BMI055_imu_param.yaml

%YAML:1.0
---
type: IMU
name: BMI055
Gyr:
   unit: " rad/s"
   avg-axis:
      gyr_n: 6.0673370376614875e-03
      gyr_w: 3.6211951458325785e-05
   x-axis:
      gyr_n: 5.4501442406047970e-03
      gyr_w: 4.0723401163659986e-05
   y-axis:
      gyr_n: 5.9380128602687073e-03
      gyr_w: 2.9388325769986972e-05
   z-axis:
      gyr_n: 6.8138540121109601e-03
      gyr_w: 3.8524127441330383e-05
Acc:
   unit: " m/s^2"
   avg-axis:
      acc_n: 3.3621979208052800e-02
      acc_w: 9.8256589971851467e-04
   x-axis:
      acc_n: 3.6095477320173631e-02
      acc_w: 9.6831827726998488e-04
   y-axis:
      acc_n: 3.4696437020780901e-02
      acc_w: 1.3092042863834673e-03
   z-axis:
      acc_n: 3.0074023283203882e-02
      acc_w: 6.7017513550209160e-04

[1] imu标定 imu_tk
[2] Imu_tk算法流程
[3] catkin_make failed #3
[4] imu_utils
[5] code_utils

camera/IMU calibration

---

roscd realsense2_camera/
roslaunch realsense2_camera rs_camera.launch
rostopic hz /camera/imu
rostopic hz /camera/color/image_raw

rosrun topic_tools throttle messages /camera/color/image_raw 20.0 /color

rosrun topic_tools throttle messages /camera/imu 200.0 /imu

Some problem:

In the rs_camera.launch, but when I check the frequency: IMU is 150 Hz and the camera is 15FPS. It can't be slow down to the frequency needed.

  <arg name="color_fps"           default="30"/>
  <arg name="gyro_fps"            default="200"/> <!-- 200 or 400-->
  <arg name="accel_fps"           default="250"/> <!--  63 or 250-->

The best frequency is 200 Hz and 30 Hz. Of course, others are still good.

rosbag record -O rs_cam15hz_imu150hz.bag /color /imu

camchain-rs_cam_hz4.yaml

cam0:
  cam_overlaps: []
  camera_model: pinhole
  distortion_coeffs: [0.3044413037380324, 2.0474157424478348, -11.061126286843251,
    18.67438520203368]
  distortion_model: equidistant
  intrinsics: [604.9671890973748, 602.1050631617551, 325.83950509989114, 238.35406753467785]
  resolution: [640, 480]
  rostopic: /color

imu.yaml

rostopic: /imu
update_rate: 150.0 #Hz
 
accelerometer_noise_density: 3.3621979208052800e-02 #continous
accelerometer_random_walk: 9.8256589971851467e-04 
gyroscope_noise_density: 6.0673370376614875e-03 #continous
gyroscope_random_walk: 3.6211951458325785e-05

roscd kalibr
cd data
cp ~/catkin_ws/src/realsense/realsense2_camera/rs_cam15hz_imu150hz.bag .
 ../python/kalibr_calibrate_imu_camera --target april_6x6_50x50cm.yaml --cam camchain-rs_cam_hz4.yaml --imu imu-BMI055.yaml --bag rs_cam15hz_imu150hz.bag 

Note that when something is wrong with the input data in bagfile, just record another one bagfile.

---

Initializing
Optimization problem initialized with 101968 design variables and 1079428 error terms
The Jacobian matrix is 2310198 x 458841
[0.0]: J: 1.35165e+06
Exception in thread block: [aslam::Exception] /home/william/kalibr_ws/src/kalibr/aslam_nonparametric_estimation/aslam_splines/src/BSplineExpressions.cpp:447: toTransformationMatrixImplementation() assert(_bufferTmin <= _time.toScalar() < _bufferTmax) failed [1.55677e+09 <= 1.55677e+09 < 1.55677e+09]: Spline Coefficient Buffer Exceeded. Set larger buffer margins!
Exception in thread block: [aslam::Exception] /home/william/kalibr_ws/src/kalibr/aslam_nonparametric_estimation/aslam_splines/src/BSplineExpressions.cpp:447: toTransformationMatrixImplementation() assert(_bufferTmin <= _time.toScalar() < _bufferTmax) failed [1.55677e+09 <= 1.55677e+09 < 1.55677e+09]: Spline Coefficient Buffer Exceeded. Set larger buffer margins!
Exception in thread block: [aslam::Exception] /home/william/kalibr_ws/src/kalibr/aslam_nonparametric_estimation/aslam_splines/src/BSplineExpressions.cpp:447: toTransformationMatrixImplementation() assert(_bufferTmin <= _time.toScalar() < _bufferTmax) failed [1.55677e+09 <= 1.55677e+09 < 1.55677e+09]: Spline Coefficient Buffer Exceeded. Set larger buffer margins!
[ERROR] [1556773048.921808]: Optimization failed!
Traceback (most recent call last):
  File "../python/kalibr_calibrate_imu_camera", line 236, in <module>
    main()
  File "../python/kalibr_calibrate_imu_camera", line 206, in main
    iCal.optimize(maxIterations=parsed.max_iter, recoverCov=parsed.recover_cov)
  File "/home/william/kalibr_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccCalibrator.py", line 179, in optimize
    raise RuntimeError("Optimization failed!")
RuntimeError: Optimization failed!

Result looks like this:

After Optimization (Results)
==================
Normalized Residuals
----------------------------
Reprojection error (cam0):     mean 0.169417479013, median 0.154212672023, std: 0.0973946838993
Gyroscope error (imu0):        mean 0.18574054756, median 0.159830346682, std: 0.115913332564
Accelerometer error (imu0):    mean 0.169497068217, median 0.145829709726, std: 0.10939033445

Residuals
----------------------------
Reprojection error (cam0) [px]:     mean 0.169417479013, median 0.154212672023, std: 0.0973946838993
Gyroscope error (imu0) [rad/s]:     mean 0.013802268496, median 0.0118768970357, std: 0.00861345010194
Accelerometer error (imu0) [m/s^2]: mean 0.0697960902289, median 0.0600502633182, std: 0.0450451310679

Transformation T_cam0_imu0 (imu0 to cam0, T_ci): 
[[ 0.01542341 -0.99976267  0.01538561  0.00713584]
 [ 0.03147917 -0.01489429 -0.99939343 -0.03487332]
 [ 0.9993854   0.01589838  0.03124198 -0.05266484]
 [ 0.          0.          0.          1.        ]]

cam0 to imu0 time: [s] (t_imu = t_cam + shift)
0.0334634768386

IMU0:
----------------------------
  Model: calibrated
  Update rate: 150.0
  Accelerometer:
    Noise density: 0.0336219792081 
    Noise density (discrete): 0.411783466011 
    Random walk: 0.000982565899719
  Gyroscope:
    Noise density: 0.00606733703766
    Noise density (discrete): 0.0743093991988 
    Random walk: 3.62119514583e-05
  T_i_b
    [[ 1.  0.  0.  0.]
     [ 0.  1.  0.  0.]
     [ 0.  0.  1.  0.]
     [ 0.  0.  0.  1.]]
  time offset with respect to IMU0: 0.0 [s]

  Saving camera chain calibration to file: camchain-imucam-rs_cam15hz_imu150hz.yaml

  Saving imu calibration to file: imu-rs_cam15hz_imu150hz.yaml
  Detailed results written to file: results-imucam-rs_cam15hz_imu150hz.txt
Generating result report...
/home/william/kalibr_ws/src/kalibr/Schweizer-Messer/sm_python/python/sm/PlotCollection.py:57: wxPyDeprecationWarning: Using deprecated class PySimpleApp. 
  app = wx.PySimpleApp()
  Report written to report-imucam-rs_cam15hz_imu150hz.pdf

References:
[1] Kalibr 标定双目内外参数以及 IMU 外参数
[2] [相机标定]RealSense D435i相机标定
[3] Problem with single imu and single cam Optimization failed #223

转载于:https://www.cnblogs.com/williamc17/p/10797677.html