superglue官方issue

Which dataset used for the released outdoor models? #46

https://github.com/magicleap/SuperGluePretrainedNetwork/issues/46

Does the released model superglue_outdoor.pth is trained under scenes be listed in the megadepth_train_scenes.txt ?

Overfitting during training on MegaDepth #81

added some random points
Is this mean
added some random ""unmatching"" points ?

points with 0 confidence

Adding random keypoints during training #90

https://github.com/magicleap/SuperGluePretrainedNetwork/issues/90

1. I indeed used a threshold of 0.005 for the detection, but dropping it to 0.001 could be safer, especially on ScanNet.
2. Yes, the matches that involve random keypoints are also used in the computation of the loss.
3. For ScanNet we used the full 640x480 resolution without any random crop.

Details for training on Megadepth #54

https://github.com/magicleap/SuperGluePretrainedNetwork/issues/54

1. These points are simply ignored when calculating the loss.
2. I think that the images were resized to a minimum size of 720 pixels during training. The thresholds of 3 and 5 pixels are for this size.

Some details about homography trained models #16

https://github.com/magicleap/SuperGluePretrainedNetwork/issues/16

1. I think that we used a threshold of 3 pixels after having resized the images to VGA 640x840.
2. I don't think that the MMA would be a suitable metric here: it essentially corresponds to the precision, which can be trivially maximized by predicting very few correct correspondences. It is only fair when used with the recall, or when the number of matches is fixed across methods, which is not possible with SuperGlue.
3. We trained with about 300k different images, but I don't think that you need so much. MS COCO, on which SuperPoint was trained, has about 80k images. What matters more is the extreme extent of the homographic and photometric augmentation

About homography pretraining for outdoor scenes #33

https://github.com/magicleap/SuperGluePretrainedNetwork/issues/33

Hi, thank you for your great work! I'm trying to train superglue+superpoint on MegaDepth dataset. As is described in the paper, the weights for outdoor scenes are initialized with homography model due to limited scene number. I'm wondering how much dose this homography pretraining affect the final results. Is it possible to obtain similar results if I train the model from scratch on megadepth or a larger outdoor dataset with more scenes?

Skydes commented on 10 Sep 2020

After further investigation it turns out that the homography pretraining is maybe not necessary if more scenes are used (with a split similar to DISK), the model is trained for longer with a slower learning rate decay, and positives and negatives are more carefully balanced. As such, I have recently obtained good results training from scratch on MegaDepth. I will be able to release more details later on.

Could you provide the training sequence numbers of MegaDepth ? #10

https://github.com/zju3dv/LoFTR/issues/10

Extent of homographies for synthetic pre-training #75

https://github.com/magicleap/SuperGluePretrainedNetwork/issues/75

Sorry for the late reply. I am not sure to understand what you mean by "limitations". The homographies that we sample, including those shown in Fig 13, are not limited to scale changes. They have large perspective transformations and some rotation. We sampled them based on hand-tuned boundary values but did not enforce a minimum number of covisible points.

Does the descriptor types limit the performance of SuperGlue? #9

https://github.com/magicleap/SuperGluePretrainedNetwork/issues/9

That's a good question :) Since SuperGlue takes raw descriptors as input, it is tailored to SuperPoint descriptors only. As such, you should retrain it if you want to match other descriptors like R2D2 or D2-Net.

What if only 50 keypoints are detected in query image? Did you add zero pad of 350 to fit the tensor shape into batch size x 400 x 256(D) ? If so, did you ignore the zero padded graph nodes when calculating loss?

https://github.com/magicleap/SuperGluePretrainedNetwork/issues/3

Thanks for your interest in SuperGlue. When fewer keypoints are detected, we sample additional ones from a uniform random distribution. We assign them a detection confidence of zero, and resample local descriptors from the semi-dense descriptor map at these locations. These are included in the matching and in the loss, and make SuperGlue robust to spurious detections. This is briefly mentioned in Section 4 of the paper.

[Discussion] Grey-scale image as an input? #2

https://github.com/magicleap/SuperGluePretrainedNetwork/issues/2

This is mostly an artifact of how the underlying SuperPoint model was trained. Like many feature detectors before it (e.g. SIFT), it operates on grayscale intensities. We found that by training on grayscale it helps improve generalization, though it seems reasonable that additional distinctiveness could come from learned color descriptors. I'd be interested to see improved results with colored SuperPoint + SuperGlue!

Is it possible to run with batch size > 1 #64

https://github.com/magicleap/SuperGluePretrainedNetwork/issues/64

Yes, SuperGlue supports batching as long as each image pair contains the same number of keypoints. SuperPoint also supports batching if the images have the same size, but can produce a different number of keypoints for each image. You would thus need to add dummy keypoints between SuperPoint and SuperGlue.

 ——Does 'keypoints1' need to the same size as 'keypoints0'?

——Just tried. Only needs to be them same size within a feature. Feature 0 and feature 1 can have different size.