【论文阅读】14-Accurate Multiple View 3D Reconstruction Using Patch-Based Stereo for Large-Scale Scenes
0 basic information
Shen S . Accurate Multiple View 3D Reconstruction Using Patch-Based Stereo for Large-Scale Scenes[J]. IEEE Transactions on Image Processing, 2013, 22(5):1901-1914.
1 characteristic
- patch-matching + refinement — accurancy
- computation --speed ???
- image parallelizing
2 method–patch-based stereo matching
2.0 patch-matching 参考& 改进
- 参考论文:
“PatchMatch Stereo - Stereo Matching with Slanted Support Windows ”
- 改进点:
- 平面表示的方式及坐标系
- 简化
(the paper VS 参考论文 )
- NCC – adjust weight neighboring pixels within
correspondence views ???- Space aggregation – space + view + temporal
- Refinement – left/right consistency
2.1 pair selection
- 从reference image i的邻居view中选择最佳的correspondding view j
reference image i --- N ( i) ------- image j
- 选择要求及量化方式:
- similar viewing direction–average [ sita (同名光线夹角)]
- suitable baseline–Distance(相机中心Ci_Cj)
- 具体操作步骤:
选择 sita满足阈值,且diatance满足关于mean(distance)的 view ,得到N(i)–neighboring views
选择 sita* distance min ,得到 best matching view: image j
2.2 depth map computaion
- supporting plane( each pixel):
基于相机坐标系
,根据3D point + normal 表示该平面 - 初始化: 大量随机(normal在一定范围内随机)—ensure get good initial at least one
(计算完image i后,imge j 的初始化可根据视图传递得到较优初值,空间传递值 vs 随机,得到较优初值)
PS :normal在一定范围内随机—–结合patch 在camera visibility(见下图)
- 平面参数优化–深度值计算
每个迭代过程
遍历每个像素,迭代更新平面参数,使得aggregation cost min 并判断是否满足阈值要求,作为有效参数估计
aggregation cost: 采用NCC (between image i and image j) ---enough for high resolution image ----相对于原论文的改进点1
迭代过程:基本同参考论文
2.3 depth refinement
核心思想:the consistency over neighboring views
- 量化指标: depth difference (measurement & projection)
Image i 中each pixel 对应X
d(X,Nk)----X与N(i)相机位置的距离–measurement,
d’(X,Nk)—将X投影到Nk的depth map上得到的深度值–projection
- refinement 的具体方式
判断d & d’差异(阈值),并根据满足要求的d &d’在N(i)中具体的个数,判断是否remove/keep
2.4 depth merging
关键点:
去除冗余(见下图)后直接project 得到 dense 3D point cloud:
3 result & other
-
和当前的方法对比: ours & PMVS & DAISY
-
Parallelizing
-
Assume memory
-
benchmark data
Fountain-P11 and Herz-Jesu-P8 -
定量化评价指标:
The depth maps( density point cloud – depth map) VS the ground truth( LIDAR SCAN triangle mesh – depth map)
-
参数分析(实验证明有效参数选择的原因)
-
实验分析每一个步骤处理的有效性(定量指标说明)
-
算法时间\speed比较