0x00 大纲
- paper: “Beyond Tracking: Selecting Memory and Refining Poses for Deep Visual Odometry” https://arxiv.org/abs/1904.01892
- code:
- 三个组件:Tracking, Memory, Refini
0x01 近期相关工作 & 需要查看的文献资料
-
联合depth和pose学习的文献[16,19,36,37,39]
-
RNN时间信息[14,22, 31–33]
-
图片不能超过5帧的原因:the high dimensionality of depth maps
-
VO在深度学习之前一直被处理成最小化几何重投影误差[10,18,20]和光度一致性误差[7,8,30];
-
Sfmlearner
是第一篇无监督学习的论文,有监督:DeMoN和DeepTAM, MapNet,DeepVO, ESP-VO,GFS-VO
(有意思,分开评估rt,lstm) -
引入相对姿态约束减少局部误差:
[4] S. Brahmbhatt, J. Gu, K. Kim, J. Hays, and J. Kautz. MapNet: Geometry-aware Learning of Maps for Camera Localization. In CVPR, 2018.
[14] G. Iyer, J. K. Murthy, K. Gunshi Gupta, and L. Paull. Geometric Consistency for Self-supervised End-to-end Visual Odometry. In CVPR Workshops, 2018.
[22] E. Parisotto, D. Singh Chaplot, J. Zhang, and R. Salakhutdinov. Global Pose Estimation with an Attention-based Recurrent Network. In CVPR Workshops, 2018. -
其他:
[32] S. Wang, R. Clark, H. Wen, and N. Trigoni. End-toend, Sequence-to-sequence Probabilistic Visual Odometry through Deep Neural Networks. IJRR, 2018.
[33] F. Xue, Q. Wang, X. Wang, W. Dong, J. Wang, and H. Zha. Guided Feature Selection for Deep Visual Odometry. In ACCV, 2018.
[5] R. Clark, S. Wang, A. Markham, N. Trigoni, and H. Wen. VidLoc: A Deep Spatio-temporal Model for 6-DoF Videoclip Relocalization. In CVPR, 2017.
The learning-based baselines include supervised approaches such as DeepVO [31], ESP-VO [32], GFS-VO [33], and unsupervised approaches such as SfmLearner [39], Depth-VO-Feat [37], GeoNet [36], Vid2Depth [19] and UndeepVO [16].
0x02 网络相关
Encoder
基于FlowNe
t,在两帧之间预测光流,输出1024个2D特征图Tracking module
包含两个模块,ConvLSTM
和SE3 layer
,前者是LSTM
的变种,LSTM
多应用在DeepVO
和ESP-VO
,ConvLSTM
保留了更多的空间信息。后者是计算两个相机移动的相对pose
,生成6-DoF
,全局的pose
计算取自于DeepVO
和ESP-VO
Memory module
使用的是经典的VO/SLAM系统,ORB-SLAM
,为了纠正ConvLSTM
不能长时间记住信息Refuning module
估计每个图片之间的绝对pose
,用的是ConvLSTM
,从这里开始看不太懂了,开始玄学看paper
这里得上下图不是很明白
0x03 作者的实验
- 数据集:KITTI [9] and TUM-RGBD [26] datasets
- encoder是在FlyingChairs dataset 预先训练好的模型
0x04 个人总结
- VISO2-M 需要研究一下,单目VO算法恢复pose。
- 是有监督得端到端的单目视觉里程计,现在正在研究的是无监督,扩展眼界意义大于实际意义;
- 引用了两个模块,Memory,Refining,前者好理解,后者一头雾水。后者还用到了
a spatial-temporal attention mechanism
- 将实验结果和经典算法还有基于学习得VO做比较
- 没有源码,很致命