使用evo工具可视化和评测SLAM算法性能

使用evo工具评测SLAM算法性能并可视化结果

1. evo tools

1.1 install evo

 - pip install  `pip install evo --upgrade --no-binary evo`
 - source install `pip install --editable . --upgrade --no-binary evo`

2. 常见格式数据集

2.1 TUM数据集格式

timestamp tx ty tz qx qy qz qw
(每行有８个元素，结尾没有空格, 时间以秒为单位)

1341847980.7900 -0.6832 2.6909 1.7373 0.0003 0.8617 -0.5072 -0.0145
1341847980.8000 -0.6829 2.6910 1.7373 0.0002 0.8613 -0.5078 -0.0146
1341847980.8100 -0.6824 2.6913 1.7372 0.0004 0.8610 -0.5084 -0.0148
1341847980.8201 -0.6821 2.6914 1.7371 0.0003 0.8609 -0.5085 -0.0151
1341847980.8301 -0.6818 2.6915 1.7371 0.0002 0.8608 -0.5086 -0.0154
1341847980.8400 -0.6815 2.6917 1.7371 0.0002 0.8611 -0.5082 -0.0158
1341847980.8500 -0.6811 2.6918 1.7371 0.0001 0.8610 -0.5084 -0.0159

2.2 EuRoC数据集

timestamp，px,py,pz,qw,qx,qy,qz,vx,vy,vz,bwx,bwy,bwz,bax,bay,baz
(每行17个元素，逗号隔开，时间以纳秒为单位)

timestamp, p_RS_R_x [m], p_RS_R_y [m], p_RS_R_z [m], q_RS_w [], q_RS_x [], q_RS_y [], q_RS_z [], v_RS_R_x [m s^-1], v_RS_R_y [m s^-1], v_RS_R_z [m s^-1], b_w_RS_S_x [rad s^-1], b_w_RS_S_y [rad s^-1], b_w_RS_S_z [rad s^-1], b_a_RS_S_x [m s^-2], b_a_RS_S_y [m s^-2], b_a_RS_S_z [m s^-2]
１403638519492829440,4.460675,-1.680515,0.579614,0.238261,-0.757610,-0.348629,-0.497711,0.006246,-0.001431,-0.004391,-0.001806,0.020940,0.076870,-0.020544,0.124837,0.061800
1403638519497829376,4.460706,-1.680523,0.579592,0.238261,-0.757616,-0.348621,-0.497709,0.006108,-0.001439,-0.004327,-0.001806,0.020940,0.076870,-0.020544,0.124837,0.061800
1403638519502829568,4.460736,-1.680530,0.579571,0.238262,-0.757621,-0.348612,-0.497706,0.006043,-0.001394,-0.004211,-0.001806,0.020940,0.076870,-0.020544,0.124837,0.061800
1403638519507829504,4.460766,-1.680536,0.579550,0.238264,-0.757625,-0.348606,-0.497703,0.005870,-0.001320,-0.004051,-0.001806,0.020940,0.076870,-0.020544,0.124837,0.061800

2.3 KiTTI数据集

r11 r12 r13 tx r21 r22 r23 ty r31 r32 r33 tz
存储变换矩阵的前三行　(每行12元素，空格隔开)　没有时间

9.999978e-01 5.272628e-04 -2.066935e-03 -4.690294e-02 -5.296506e-04 9.999992e-01 -1.154865e-03 -2.839928e-02 2.066324e-03 1.155958e-03 9.999971e-01 8.586941e-01
9.999910e-01 1.048972e-03 -4.131348e-03 -9.374345e-02 -1.058514e-03 9.999968e-01 -2.308104e-03 -5.676064e-02 4.128913e-03 2.312456e-03 9.999887e-01 1.716275e+00
9.999796e-01 1.566466e-03 -6.198571e-03 -1.406429e-01 -1.587952e-03 9.999927e-01 -3.462706e-03 -8.515762e-02 6.193102e-03 3.472479e-03 9.999747e-01 2.574964e+00
9.999637e-01 2.078471e-03 -8.263498e-03 -1.874858e-01 -2.116664e-03 9.999871e-01 -4.615826e-03 -1.135202e-01 8.253797e-03 4.633149e-03 9.999551e-01 3.432648e+00
9.999433e-01 2.586172e-03 -1.033094e-02 -2.343818e-01 -2.645881e-03 9.999798e-01 -5.770163e-03 -1.419150e-01 1.031581e-02 5.797170e-03 9.999299e-01 4.291335e+00
9.999184e-01 3.088363e-03 -1.239599e-02 -2.812195e-01 -3.174350e-03 9.999710e-01 -6.922975e-03 -1.702743e-01 1.237425e-02 6.961759e-03 9.998991e-01 5.148987e+00
9.998890e-01 3.586305e-03 -1.446384e-02 -3.281178e-01 -3.703403e-03 9.999605e-01 -8.077186e-03 -1.986703e-01 1.443430e-02 8.129853e-03 9.998627e-01 6.007777e+00
9.998551e-01 4.078705e-03 -1.652913e-02 -3.749547e-01 -4.231669e-03 9.999484e-01 -9.229794e-03 -2.270290e-01 1.649063e-02 9.298401e-03 9.998207e-01 6.865477e+00

2.4 数据格式的转换

"把EuRoc的数据格式转化成TUM数据格式"
evo_traj  euroc /home/ubuntu/dataset/EuRoc/MH-05/state_groundtruth_estimate0/data.csv 　--save_as_tum

"TUM数据转换成kitti格式"
evo_traj tum traj_1.txt 　traj_2.txt 　traj_3.txt 　--save_as_kitti

" 把一些文件的数据格式转化成rosbag"
evo_traj tum traj_1.txt traj_2.txt traj_3.txt --save_as_bag

参考文档: https://github.com/MichaelGrupp/evo/wiki/Formats

3. 评测流程

3.1. 查看单条轨迹基本信息

evo_traj euroc  MH_01_gt.csv --plot
evo_traj tum    MH_01_vio.csv --plot

通过这种方式可以看到估计的结果中包含多少 camera pose, path的大致长度，轨迹的持续时间等
　infos: 36382 poses, 80.626m path length, 181.905s duration

3.2 两条轨迹的比较

3.３. 与GT比较进行误差分析

evo_ape 绝对位姿误差，一般评价轨迹的全局一致性用
Absolute pose error, often used as absolute trajectory error. Corresponding poses are directly compared between estimate and reference given a pose relation. Then, statistics for the whole trajectory are calculated. This is useful to test the global consistency of a trajectory.
　evo_rpe　相对位姿偏差
Instead of a direct comparison of absolute poses, the relative pose error compares motions (“pose deltas”). This metric gives insights about the local accuracy, i.e. the drift. For example, the translational or rotational drift per meter can be evaluated

evo_ape euroc Data/EuRoC/V1_01_easy/state_groundtruth_estimate0/data2.csv Data/EuRoC/CameraTrajectory.txt  -va --plot --save_results results/ORB.zip

"tum 格式误差比较"
evo_rpe tum reference.txt estimate.txt --pose_relation angle_deg --delta 1 --delta_unit m

3.3

3.4 evo使用技巧

把配置文件导出到文件，以便下次调用直接使用

"对齐　画图　比较"
evo_config generate --pose_relation angle_deg --align --correct_scale  --delta 1 --delta_unit m --verbose --plot --out rpe_config.json

"利用配置文件画图"
evo_rpe tum  MH_01_gt.tum vio.csv -c rpe_config.json

"画图　并将结果保存到本地"
evo_rpe tum  MH_01_gt.tum vio.csv -c rpe_config.json --save_result MH_01.zip

常用的evo 命令选项

• –align or -a = SE(3) Umeyama alignment (rotation, translation)
• –align --correct_scale or -as = Sim(3) Umeyama alignment (rotation, translation, scale)
• –correct_scale or -s = scale correction only
• –plot_mode=xz　选择可是几维空间

改变 evo_config设置，让图画的更美观

"查看配置"
evo_config show

"修改配置"
env_config set ***
plot_seaborn_style whitegrid //修改背景栅格　默认 darkgrid
plot_seaborn_palette bright　//修改线条颜色　bright、deep6、colorblind、pastel、dark、muted 默认deep6
plot_linewidth 1.5　//修改线宽
plot_reference_color // 默认black 

参考文档：https://github.com/MichaelGrupp/evo/wiki/Plotting

4. 需要注意的问题

evo_traj euroc MH_01_gt.csv --save_as_tum
evo_traj tum vio.csv --ref=MH_03_gt.tum -c rpe_config.json
evo_rpe tum MH_03_gt.tum vio.csv -c rpe_config.json --save_result MH_03.zip
evo_res *.zip -p --serialize_plot all_plot --use_filenames --save_table table.csv

evo_rpe kitti kitti_00_gt.txt vio.txt -va --align --correct_scale --plot --plot_mode xz --save_results kitti_00_sequence

evo_rpe tum MH_02_gt.tum vio.csv -va --align --plot --plot_mode xyz --save_results MH_02_sequence

测量绝对误差，这个才是大多数中论文的指标
evo_ape tum V1_03_gt.tum vio.csv -va --align --plot --plot_mode xyz --save_results V1_03_sequence

时间戳同步问题