openMVS-- Resrtuction Mesh (原理及代码解读)
1、 pre-process
- reset image resolution to the original size
经过densify后生成的scene数据保存有point对应的image信息,在densify过程中,reference imgae与neighboring image 做过尺度统一的处理(可参考作者另一篇博文??),因此需要reset solution
- initializing image neighbors
- 权重设置
- 稠密重建后,每个三维点对应了一个权重array,保存三维点与对应的可见图像间的权重
- 代码
scene.pointcloud.pointWeights.Release()
bUseConstantWeight //considers all view weights 1 instead of the available weight
2、Resrtuction mesh
- 核心函数
bool Scene::ReconstructMesh(float distInsert, bool bUseFreeSpaceSupport, unsigned nItersFixNonManifold,
float kSigma, float kQual, float kb,
float kf, float kRel, float kAbs, float kOutl,
float kInf
)
- 函数实现参考论文:
Exploiting Visibility Information in
Surface Reconstruction to Preserve Weakly Supported Surfaces
代码阅读笔记见作者另一篇博文 :链接
2.1、 DT – Incrementally
2.1.1、insert 3D point (包含alpha_vis(P)的计算)
- !!判断能否将该点有效插入的条件:
O && 已构建的DT的顶点P的重投影距离—阈值( 代码中为:distInsert^2)比较:
若不满足,则不添加该点,但visibility info更新(alpha(3D point) +1;
若不满足,则insert该点,visibility info初始化(alpha(3D point)=1)
重投影: 将O与P重投影到所有与o相关的视图上,
只要有一个视图
满足条件far enough即可
- 具体步骤如下:
---------------------搜寻需要与O判断的P------------------------------
1). locate O –cell(DT)
const cell_handle_t c(delaunay.locate(p, lt, li, lj, hint->cell()));
---------------------------------
// Returns the (finite or infinite) cell p lies in.
// Starts at cell "start".
// If lt == OUTSIDE_CONVEX_HULL, li is the index of a facet separating p
// from the rest of the triangulation
// In dimension 2 :
// returns a facet (Cell_handle,li) if lt == FACET
// returns an edge (Cell_handle,li,lj) if lt == EDGE
// returns a vertex (Cell_handle,li) if lt == VERTEX
// If lt == OUTSIDE_CONVEX_HULL, li, lj gives the edge of c separating p
// from the rest of the triangulation
// lt = OUTSIDE_AFFINE_HULL if p is not coplanar with the triangulation
exact_locate(const Point& p, Locate_type& lt, int& li, int& lj,
Cell_handle start, bool *could_lock_zone)
2). cell 中离O最近的vertex –P(1个)
nearest = delaunay.nearest_vertex_in_cell(p, c);
---------------------搜寻需要与O判断的P------------------------------
3). 重投影判断
if (!IsDepthSimilar(pn.z, pe.z) || normSq(Point2f(pn)-Point2f(pe)) > distInsertSq)
4). 更新visibility info
hint->info().InsertViews(pointcloud, idx);
InsertViews()
函数内部实现 (二选一):
- view_t初始化:
views.InsertAt(idx, view_t(viewID, weight)); //将3D point O 的可见视图深入到view_t( 带有weight)
- view_t自增:
views[idx].weight += weight;//weight = 1
struct view_t {
PointCloud::View idxView; // view index
Type weight; // point's weight
……}
- insert 操作代码:
delaunay.insert( p)
2.1.2、finite\infinite face\DT(DT-s_weight)
PRE:
- DT生成过程,vertex只包含3D point,生成的DT包括有限DT +无限DT,通过相机视锥体与无限DT所决定的有限face,以及store the finite facet of the infinite cells,使整个face覆盖的空间为3D point + Camera
(对应操作如下1 && 2)
- 同时,将包含相机的DT视作outside,初始化该DT的s-edge weight
(对应操作如下:3)
//store the finite facet of the infinite cells
hullFacets.push_back(facet);
// Given a cell and a camera inside it, if the cell is infinite,
// find all facets on the convex-hull and inside the camera frustum,
// else return all four cell's facets
template <int FacetOrientation>
void fetchCellFacets(const delaunay_t& Tr, const std::vector<facet_t>& hullFacets, const cell_handle_t& cell, const Image& imageData, std::vector<facet_t>& facets)
// link all cells contained by the camera to the source ---outside!!!
infoCells[f.first->info()].s = kInf; ----DT-s 的权重已经在此设置,为常数kInf
重要:
- 并不是所有DT都与需要设置DT-s权重,只有
由camera fetch得到的face,与其相关的DT(代码如下)
设置该权重本质:
部分置信度极低的DT-s edge已经剔除,置信度较高的DT才生成DT-s edge
camCell.cell = delaunay.locate(MVS2CGAL(camera.C));
2.2、 weight【weight_vis && weight_qual】 (DT-DT //DT-t)计算
2.2.0、pre
- alpha_vis(P)在DT生成过程(incrementally)就已实现,
- 遍历P及其可见视图对应的相机camera,根据(c,p)计算对应DT-t及DT-DT的weight_vis_p 最后累加得到weight_vis
- weight_vis的计算基于 alpha_vis(P) 以及distance (与参考论文一致)
- weight_qual 只有DT-DT才需要计算
- 重要参数-sigma 计算:all DT edge length
media
2.2.1、 Camera-vertex之间(DT-DT weight)
2.2.1.0、pre
遍历(c,p)间或者延长线间的DT face,核心函数intersect(),会输出out_face,即 所有in_face的邻接DT的其他face,从而实现遍历作用
2.2.1.1 weight_vis
- 原理:每一个与ray相交的面得到与此(c,p)相关的权重:(与参考论文一致)
- 代码:
const edge_cap_t alpha_vis(view.weight);
//view.weight 在DT三角化过程中计算得到(见上)
const edge_cap_t w(alpha_vis*(1.f-EXP(-SQUARE((float)inter.dist)*inv2SigmaSq)))
2.2.1.2 weight_qual
在graph构建的过程中,只有DT间face所代表的edge的权重,需要引入surface quality,与可视性权重加权【kQual】相加---与参考论文一致
- 代码实现,见 2.4
2.2.2、Camera-vertex –to end(DT-DT/DT-t weight)
- Camera-vertex –to end: 沿可视光线延长到p后方,计算与L_DT(
endCell
)相关的权重(L_DT的定义见参考论文)
const cell_handle_t endCell(delaunay.locate(segEndPoint.source(), vi->cell()));
- 对于L_DT -t 的权重,直接把所有将该DT作为L_DT 的相关(c,p) 的
alpha_vis
累加
t += alpha_vis;
- DT-DT weight:对于L_DT 的相交face的权重,采用上文(2.2.1)的方法进行计算
2.3、 add free-space support to the t-edge (graph的边)weight
- 计算f(T):原理 –同参考论文
//代码实现,基于f(face),核心函数如下:
// Given a cell, compute the free-space support for it
edge_cap_t freeSpaceSupport(const delaunay_t& Tr, const std::vector<cell_info_t>& infoCells, const cell_handle_t& cell)
- 计算(c,p)与free-space support相关的指标:
- beta:
- gamma
- eps
- 运用论文中的point interface classifier 判断并更新 t-edge weight
PS:代码是直接累乘,和参考论文不一致
2.4、 s-t graph cut —mesh
- 在graph构建的过程中,只有DT间face所代表的edge的权重,需要引入surface quality,与可视性权重加权【kQual】相加—参考论文1
- 代码:
const edge_cap_t q((1.f - MINF(computePlaneSphereAngle(delaunay, facet_t(ci,i)), computePlaneSphereAngle(delaunay, facet_t(cj,j))))*kQual);
graph.AddEdge(ciID, cjID, ciInfo.f[i]+q, cjInfo.f[j]+q);
3、 non-manifold 迭代优化(略)
4、 Mesh -clean
- clean 阶段调用scene.mesh.Clean()函数-3次
- scene.mesh.Clean()函数核心内容: