深度估计中的不确定度方法总结

针对深度估计、深度补全和稀疏重建等深度估计相关任务，需要利用不确定度或置信度来评价估计结果的好坏。本文总结了一些基于不确定度进行辅助和评价结果可靠性的有效方法。

📚***Uncertainty-Aware CNNs for Depth Completion: Uncertainty from Beginning to End,提出了自监督的概率归一化的卷积网络，同时预测深度与不确定度。一方面估计了输入数据的不确定度，使得网络可以基于数据可靠性进行针对性的学习；另一方面提出了概率化的NCNN，将训练过程转变为最大化似然估计问题，实现了输出不确定度的估计。通过归一化卷积操作，将信号中置信概率高的投影到子空间中，而后再重新利用函数计算出置信度低的部分投影回正常空间中。随后通过置信度传播的方法在估计出深度ri的同时也得到了最终输出的置信度。将置信度用于编码重建过程中的方差得到si(图中的大C为损失函数)，随后与ri一同得到最终估计出的结果yi。这种方法得到的不确定度具有语义信息，与误差图相似可以解释。(from 瑞典林雪平大学)

ref： intermediate confidence mask filter out noisy datas：
28.Sparse and noisy lidar completion with rgb guidance and uncertainty. In 2019 16th International Conference on Machine Vision Applications (MVA), pages 1–6. IEEE, 2019
33.Depth completion from sparse lidar data with depth-normal constraints. In The IEEE International Conference on Computer Vision (ICCV), October 2019
36.Deep surface normal guided depth prediction for outdoor scene from sparse lidar data and single color image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3313–3322, 2019.

Bayes Deep Learining:
12.Representing model uncertainty in deep learning. In international conference on machine learning, pages 1050–1059, 2016
9.Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems, pages 6402–6413, 2017.
20.Evaluating scalable bayesian deep learning ¨ methods for robust computer vision. arXiv preprint arXiv:1906.01620, 2019
24.Self-supervised sparse-to-dense: self-supervised depth completion from lidar and monocular camera. In 2019 International Conference on Robotics and Automation (ICRA), pages 3288–3295. IEEE, 2019

sparse data and confidence:
7.Propagating confidences through cnns for sparse data regression. In The British Machine Vision Conference (BMVC), Northumbria University, Newcastle upon Tyne, England, UK, 3-6 September, 2018, 2018
8.Confidence propagation through cnns for guided sparse depth regression. IEEE transactions on pattern analysis and machine intelligence, 2019

📚***D3VO: Deep Depth, Deep Pose and Deep Uncertaintyfor Monocular Visual Odometry 预测像素的后验概率 由像素估计得到的图像不确定度， 详见3.1部分公式6(from 慕尼黑工大)

probability based uncertainty ref：
35.[What uncertainties do we need in bayesian deep learning for computer vision?](https://arxiv.org/pdf/1703.04977.pdf)
36.[Multi-task learning using uncertainty to weigh losses for scene geometry and semantics](https://arxiv.org/abs/1705.07115) ---[code](https://github.com/ranandalon/mtl)
40.[Supervising the new with the old: learning SFM from SFM](https://openaccess.thecvf.com/content_ECCV_2018/papers/Maria_Klodt_Supervising_the_new_ECCV_2018_paper.pdf)

📚***On the uncertainty of self-supervised monocular depth estimation,实现不确定度的方法主要有以下几种：利用了翻转图片输入的方法来获得不确定度；利用网络采样test dropout，随机子数据集训练多个子网络，训练过程中多个模型快照叠加；预测方式估计不确定度，包括学习重投影和最大化条件后验概率，利用教学网络，将另一个子网络去模拟主网络生成结果的分布，使其生成不确定度。所以本文利用了两个网络，一个做重建，另一个来模型这个重建网络的生成，随后得到了学生网络用自监督的方式，模拟前一个网络生成的分布（from 意大利博洛尼亚大学）

ref:code
model-inherent，基于内部流模型估计不确定度：
7.* A confidence measure for variational optic flow methods
36.Bootstrap optical flow confidence and uncertainty measure
71.**ProbFlow: Joint Optical Flow and Uncertainty Estimation
31.Laf-net: Locally adaptive fusion networks for stereo confidence estimation,lab

posthoc，后处理方式分析已经估计好的场
33.An adaptive confidence measure for optical flows based on linear subspace projections
34.*A statistical confidence measure for optical flows
43.**Learning a confidence measure for optical flow,projects

贝叶斯方法，采样权重分布来得到模型，估计方差和均值：
10.Stochastic gradient hamiltonian monte carlo.
44.A practical bayesian framework for backpropagation networks
73.Bayesian learning via stochastic gradient langevin dynamics.

变分推理的方法代替采样：
6.贝叶斯网络Weight uncertainty in neural networks.

21.*Practical variational inference for neural networks.

直接估计不确定性，直接利用网络预测每一点的均值和方差而不去采样，代替单个值，十分高效不需要多次前传
32.Supervising the new with the old: learning sfm from sfm
49.Estimating the mean and variance of the target probability distribution.

LAF-Net: Locally Adaptive Fusion Networks for Stereo Confidence Estimation，从GT置信图中学习出来的置信度。(from 韩国延世大学)

ref： 基于双目的损失函数做的confidence map,*双目深度估计的视差图置信度有自己的定义方法* [Leveraging Stereo Matching With Learning-Based Confidence Measures](https://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Park_Leveraging_Stereo_Matching_2015_CVPR_paper.pdf) [Learning to Detect Ground Control Points for Improving the Accuracy of Stereo Matching](https://www.cv-foundation.org/openaccess/content_cvpr_2014/papers/Spyropoulos_Learning_to_Detect_2014_CVPR_paper.pdf) [Learning from scratch a confidence measure](https://vision.disi.unibo.it/~mpoggi/papers/bmvc2016.pdf), [课题组](https://vision.disi.unibo.it/~mpoggi/code.html) [code](https://github.com/fabiotosi92/Unsupervised-Confidence-Measures)

---

pic from pexels.com