【论文阅读】13-Multi-View Stereo for Community Photo Collections

部分参考：链接

0 basic information

Goesele, Michael, Snaveley, Noah, Curless, Brian,等. Multi-View Stereo for Community Photo Collections[J]. Proc.int.conf.on Computer Vision, 2007:1-8.

1 background

CPCs（community photo collections）数据 特点：

2 method

2.0 Calibrating Internet Photos

1. geometrically
   radial distortion(径向畸变)
   内参、外参
2. radiometrically

2.1 Estimate depth maps

2.1.1 Find good matches ----重要！

2.1.1.1 image level—global view selection

1. 目的：find a set of good neighborhood images N ----(usually |N| = 10)
2. good ：sufficient parallax(视差)
3. good 影响因素：

• shared feature points number
• baseline length
• scale consistency

4. 量化形式：
   

---

考虑 feature number & baseline( angle)：

---

考虑 feature number & scale

5. N 的生成方式： greedy approach

grow the neighborhood incrementally by iteratively adding to N the
highest scoring view

2.1.1.2 !!Rescaling Views —after global selection

当对不同的reference view 进行深度估计时，neighbor views 根据计算进行与reference view 相对应的scale操作
原因：CPC图像集本身具有较大的尺度差异

1. Resolution_lowest — Vmin
   相对尺度\分辨率计算公式如下：
   实质： 平均-Shared feature points–average
   
   

2. R to Resolution_lowest—R_rescaled

• threshold t(min the relative scale)！！！
• rescale R!!!

3. N-Vmin to R_rescaled

4. NOTICE: rescale with different reference view
   

2.1.1.3 pixel level —local view selection A

1.基于pixel
每个pixel 对应不同的A，并且neighboring image set = A在立体匹配迭代的过程中会不断更新
2.选择要求及定量化指标：

• corresponding pixels photometrically consistent --mean-removed normalized cross correlation(NCC)
• sufficiently wide range of observation directions— span of directions from which a given scene point— angular spread

angular spread: the angle of epipolar lines obtained by projecting
each viewing ray passing through the scene point into the reference
view.(该像素的多个候选neighboring view的极线在reference view 上的投影相互之间形成的夹角 )
将其作为score_local的计算因子

ps：极线—根据F计算

3. 选择的步骤：

• Highest score LR(V)
• NCC higher threshold ----add to A
• Repeat stop： the size of A / N no remain

2.1.2 optimizing for both depth and normal

2.1.2.1 image pixels priori queen Q --initial

1.3D POINMTS :

sfm sparse points 
+
addition points: feature points visible in all N views

2. project 3D POINTS to reference image R—pixels queen Q

2.1.2.2 visit each pixel in queen Q—pixel in R & A ,实现立体匹配

• traversal --prioritized–estimated matching confidence --selecting the highest one
• each selected pixel

1. Pixel—window—planar patch in scene
2. Planar patch 参数化表示—center pixel的深度h、邻域像素在s\t方向上的单位深度偏移量hs、ht—(ps:hs ht 可表示normal)
3. Planar patch —project to A—window’
4. 基于Window& window’一致性约束—构建方程并优化目标函数求解未知量

• 约束：根据反射模型（选择简单的Lambertian reflectance），匹配位置处的像素值(RGB三通道)对应的感光强度（入射光线强度）相差一个color factor -ck(不同view、不同通道不同一个ck)

• 目标函数：SSD
• 方程个数： 3(RGB) * n * n(window size)*m(A size)
• 未知量：(center pixel -h \hs\ ht) 3+ 3*m （ck number）

5. 计算depth 、normal的confidence：average of mean-remove NCC in neighboring view
6. 迭代优化计算细节----见原文，此处略，迭代过程中A更新

3 other

MVS benchmark datasets
MVS evaluation method