基于相片集的头发重建研究实现

        The overall flow is as described as follows. Firstly, we use colmap to get the frame pointcloud. After that, though hair segmentation with deep learning and gabor filter, we obtain the 2D hair direction. Finally, according to the frame pointcloud which has 3D points for all pixels, we can form the 3D hair fibers. Collecting these 3D hair fibers together , the hair object is generated at last with some refinements.

    

一 点云重建以及处理过程

    点云重建原理

   使用colmap (https://demuc.de/colmap/#documentation)， 而colmap 的原理，是基于SFM 以及 MVS。

    多视角图像 -> 图像特征提取匹配 -> 稀疏重建(SFM) -> 稠密重建(MVS) -> 点云模型化 -> 三维模型

    SFM ： detect 2D features in images -> match 2d features between images ->  construct 2D tracks from matches

                   ->solve for SFM model from the 2D tracks -> refine SFM model using bundle adjustment

 

 

二 图片头发分割 和 头发纹理提取()

1 基于深度学习的 hair sagmentation (https://github.com/akirasosa/mobile-semantic-segmentation)

  参考 MobileNetV2: Inverted Residuals and Linear Bottlenecks， 使用LFW 作为训练集。

   https://blog.csdn.net/mzpmzk/article/details/82976871 此连接，有关于MobileNetV2 的较为详细的介绍。

2 Gabor 滤波

   gabor 滤波是为了得到 头发的纹理(2D direction) 信息，根据这个2D点在图片中的坐标，结合点云，可以得到头发的3D坐标。从而得到头发 OBJ 

 

三 头发部分的最后生成以及渲染( 参考前2篇博文)

 

 

Reference：

Fingerprint Enhancement using Improved Orientation and Circular Gabor Filter. Avinash Pokhriyal., Praveen Sengar

A Data-driven Approach to Four-view Image-based Hair Modeling. Meng Zhang, Menglei Chai. Transactions on Graphics

Modeling Hair from Multiple Views. Yichen Wei, Eyal Ofek, etc,.

Avatar Digitization From a Single Image For Real-Time Rendering. Liwen Hu, Shunsuke Saito, etc,. ACM transactions on Graphics, Vol. 36, No.6, Article, 1