ICRA2020开源项目DF-VO的BUG笔记

前言

害，我本来是Ubuntu18+win10的双系统的，结果昨天在装软件的时候直接把ubuntu搞崩了，直接很多软件被删掉了。加上我本身很讨厌ubuntu18的个别设定，索性直接重装了Ubuntu20。

因为要运行DF-VO，所以要安装相应的环境，然而当我装好后，发现有一个bug各种不能解决，解决玩之后运行起来，但是效果完全不一样。各种方式都试过了，就是跟Ubuntu18的效果不一样。通过各种调参，也加深了理解，虽然部分效果很一般，但是整体还是可以的。

现在是真的很糟心

BUG

2.1.2 上来就跟我说没有“motion”

DF-VO项目中的dfvo.py文件中第158行

pose = self.ref_data['motion']

报错如下：

KeyError: 'motion'

这个部分是我最糟心的，其产生的原因我也表示无法理解

为了解决这个问题，我特意回去看了一下他的论文，这部分是生成关键点匹配的，然后这个报错是因为没有关键点匹配，所以字典没有这个Key

这时候我又去看他的源码，源码中default_configuration.yml文件中匹配的部分，他有两种基本的匹配模式，内容如下：

#-------------------------------------
#- Correspondence (keypoint) selection
#-------------------------------------
kp_selection:                                             # correspondence selection configuration
    local_bestN:                                          # local best-N configuration
        enable: True                                      # enable/disable local best-N selection
        num_bestN: 2000                                   # number of keypoints
        num_row: 10                                       # number of divided rows
        num_col: 10                                       # number of divided columns
        score_method: flow                                # selection score, [flow, flow_ratio]
        thre: 0.1                                         # flow consistency masking threshold
    bestN:
        enable: False                                     # enable/disable best-N selection
        num_bestN: 2000                                   # number of keypoints
    sampled_kp:                                           # random/uniform keypoint sampling
        enable: False                                     # enable/disable random/uniform keypoint sampling
        num_kp: 2000                                      # number of keypoints to be extracted
    rigid_flow_kp:                                        # keypoint selection from optical-rigid flow consistency (for scale recovery)
        enable: False                                     # enable/disable rigid-flow based keypoint selection
        num_bestN: 2000                                   # number of keypoints
        num_row: 10                                       # number of divided rows
        num_col: 10                                       # number of divided columns
        score_method: opt_flow                            # selection score, [rigid_flow, opt_flow]
        rigid_flow_thre: 5                                # masking threshold for rigid-optical flow consistency 
        optical_flow_thre: 0.1                            # masking threshold for forward-backward flow consistency 
    depth_consistency:                                    # (Experiement Ver. only) depth consistency configuration
        enable: False                                     # enable/disable depth consistency
        thre: 0.05                                        # masking threshold

2.1.2 上来就跟我说没有“motion”——解决办法

默认的匹配选的是local_bestN，然后我就换了一下匹配方式，改成bestN，然后就可以了

更换后的如下：

#-------------------------------------
#- Correspondence (keypoint) selection
#-------------------------------------
kp_selection:                                             # correspondence selection configuration
    local_bestN:                                          # local best-N configuration
        enable: False                                     # enable/disable local best-N selection
        num_bestN: 2000                                   # number of keypoints
        num_row: 10                                       # number of divided rows
        num_col: 10                                       # number of divided columns
        score_method: flow                                # selection score, [flow, flow_ratio]
        thre: 0.1                                         # flow consistency masking threshold
    bestN:
        enable: True                                     # enable/disable best-N selection
        num_bestN: 2000                                   # number of keypoints
    sampled_kp:                                           # random/uniform keypoint sampling
        enable: False                                     # enable/disable random/uniform keypoint sampling
        num_kp: 2000                                      # number of keypoints to be extracted
    rigid_flow_kp:                                        # keypoint selection from optical-rigid flow consistency (for scale recovery)
        enable: False                                     # enable/disable rigid-flow based keypoint selection
        num_bestN: 2000                                   # number of keypoints
        num_row: 10                                       # number of divided rows
        num_col: 10                                       # number of divided columns
        score_method: opt_flow                            # selection score, [rigid_flow, opt_flow]
        rigid_flow_thre: 5                                # masking threshold for rigid-optical flow consistency 
        optical_flow_thre: 0.1                            # masking threshold for forward-backward flow consistency 
    depth_consistency:                                    # (Experiement Ver. only) depth consistency configuration
        enable: False                                     # enable/disable depth consistency
        thre: 0.05                                        # masking threshold

2.2.1光流正向反向流动一致性趋于缩小

这个问题怎么说呢，就是导致上一个bug的罪魁祸首，因为光流正向反向流动一致性趋于缩小，导致匹配空间缩小，导致用local_bestN方法无法匹配

能运行之后发现没有，右下角的图很奇怪，根本不像论文里展示的，可匹配空间贼小，就是因为这个才导致的上一个BUG，害，到这真的贼糟心，明明是同一个预训练模型，同一个代码

右上角几张图，就是匹配的keypiont，只集中在中间，就是因为用了bestN方法

2.2.2光流正向反向流动一致性趋于缩小——解决办法

其实这里也不能叫解决吧，只能说缓解，可能我看着有效果而已

我把flow网络的微调打开了，就感觉稍微要好一点点，也可能是我的错觉

参数如下：

# ------------------------------------
# Online Finetuning
# ------------------------------------
online_finetune:                                          # online fine-tuning configuration
    enable: True                                         # enable/disable flow finetuning
    save_model: False
    lr: 0.00001                                           # learning rate
    num_frames: 200                                           # number of frames to be fine-tuned, [None, int]
    flow:                                                 # flow fine-tuning configuration
        enable: True                                     # enable/disable flow finetuning
        scales: [1, 2, 3, 4, 5]                           # scales to be used for training
        loss:                                             # flow loss configuration
            flow_consistency: 0.005                       # forward-backward flow consistency loss weight
            flow_smoothness: 0.1                          # flow smoothness loss weight
    depth:                                                # depth fine-tuning configuration
        enable: False                                     # enable/disable depth finetuning
        scales: [0, 1, 2, 3]                              # scales to be used for training
        pose_src: DF-VO                                   # pose source for depth-pose finetuning [DF-Vo, deep_pose]
        loss:                                             # depth loss configuration
            apperance_loss: 1                             # apperance loss weight
            disparity_smoothness: 0.001                   # disparity smoothness loss weight
            depth_consistency: 0.001                      # depth consistency loss weight
    pose:                                                 # pose finetuning configuration (with depth)                  
        enable: False                                     # enable/disable pose finetuning
    
    stereo:
        enable: False

效果如下

我觉得可匹配空间要大一些

2.3 效果完全不一样

整个效果不一样，我觉得都是因为flow网络引起的，我现在还是想不明白，为什么我换了一个环境，网络输出会不一样，而导致后面那么多bug，越想越不能接受昨晚自己系统崩了的事实，害~

如果知道根本原因的，可以留言告诉我嘛，感谢