人脸对齐--Face Alignment In-the-Wild: A Survey

Face Alignment In-the-Wild: A Survey
Computer Vision and Image Understanding Volume 162, September 2017, Pages 1-22
https://www.sciencedirect.com/science/article/pii/S1077314217301455

人脸对齐综述文献

还有另一篇总数文献
Facial feature point detection: A comprehensive survey
Neurocomputing Available online 1 June 2017
http://www.sciencedirect.com/science/article/pii/S0925231217308202

人脸对齐的定义： face alignment can be formulated as a problem of searching over a face image for the pre-defined facial points (also called face shape), which typically starts from a coarse initial shape, and proceeds by refining the shape estimate step by step until convergence。During the search process, two different sources of information are typically used: facial appearance and shape information. The latter aims to explicitly model the spatial relations between the locations
of facial points to ensure that the estimated facial points can form a valid face shape.
人脸对齐可以看作在一张人脸图像搜索人脸预先定义的点（也叫人脸形状），通常从一个粗估计的形状开始，然后通过迭代来细化形状的估计。在搜索的过程中，两种不同的信息被使用，一个是人脸的外观 appearance ，另一个是形状。形状提供一个搜索空间上的约束条件

人脸对齐主要讲人脸中的 eyes, mouth, nose and chin 检测出来，用特征点标记出来

人脸对齐是一个中间步骤，首先是人脸检测，然后是人脸对齐，人脸对齐的结果可以用于：
人脸识别、Face recognition，属性计算、Attribute computing，表情识别、Expression recognition

人脸对齐的挑战性还是比较大的

这里我们首先将人脸对齐算法分为两大类： generative and discriminative
1） Generative methods 这类方法对人脸形状和外观都建立 generative models，将人脸对齐问题看作一个搜索 shape and appearance 参数的优化问题

2）Discriminative methods 这类方法 直接分别训练特征点检测器，这些点的位置受到形状的约束

下面我们来深入这两类方法

3 Generative methods
因为人脸的形状和外观多变性，通常我们将人脸建模为 deformable objects。用于人脸对齐的 Generative methods 对人脸外观构建 parametric models，拟合生成模型的目标是找到一组形状和外观参数可以得到一个最符合测试人脸图像的生成模型。基于人脸特征表示的类型，生成模型方法可以进一步细分为基于整体表示的 Active Appearance Models (AAMs) 和基于局部表示的 part-based generative deformable models

3.1.1. Basic AAM algorithm: modeling and fitting
基本的 Active Appearance Models (AAMs) 算法包括 modeling and fitting 两个部分：
AAM modeling包括 shape model, appearance model, and motion model

AAM fitting 这个拟合主要是寻找测试图像 test image 和 参考图像 reference image 之间的映射关系

3.1.2. Recent advances on AAMs
近年来针对AAMs的改进主要在三个方面：
(1) unconstrained training data [22], 现实场景的训练数据
(2) robust image representations [75, 76] 新的特征表示方法
(3) robust and fast fitting strategies [75, 22] 新的拟合算法

3.1.3 Discussion
AAMs 的缺点: 1)因为 holistic appearance model 被使用，局部遮挡情况难以被解决 2）appearance parameter 维数很高，导致难优化，容易收敛于局部极小值

3.2. Part-based generative deformable models
Part-based generative methods build generative appearance models for facial parts, typically with a shape model to govern the deformations of the face shapes
局部生成方法对人脸局部特征建立 generative appearance models，并使用一个 shape model 分析人脸形状的变形

一般来说有两类方法构建 generative part models：一个是对每个人脸局部特征构建独立的 appearance model，另一个是 对所有的人脸局部特征同时构建 generative models

Part-based generative methods 优点：
more robust to global lighting and occlusion in wild conditions， easier to optimize

随着现实场景训练数据的增加， discriminative methods 在人脸对齐中的优势比较明显

4 Discriminative methods
Discriminative face alignment methods seek to learn a (or a set of) discriminative function that directly maps the facial appearance to the target facial points
这里有分两条技术线路：
1）The first line is to follow the “divide and conquer”strategy by learning discriminative local appearance model (detector or regressor) for each facial point, and a shape model to impose global constraints on these local models
2）The second line is to directly learn a vectorial regression function to infer the whole face shape, during which the shape constraint is implicitly encoded.

4.7. Summary and discussion
CLMs, constrained local regression and DPMs follow the “divide and conquer” principle to simplify the face alignment task by constructing individual local appearance model for each facial point
这类方法效果不是很好，于是有了另一条研究线路
another main stream in face alignment is to jointly estimate the whole face shape from image, implicitly exploiting the spatial constraints among facial points

5 Towards the development of a robust face alignment system
这里先提出一个整体框架，然后再优化细节问题

5.2. Training data augmentation
数据增强还是很重要的，这里介绍了四个方法

5.3. Face preprocessing
For the task of face alignment, it is useful to remove the scaling variations of the detected faces, and enlarge the face region to ensure that all predefined facial points are enclosed

5.3.1. Handling scaling variations
rescaling the bounding box produced by the face detector
5.3.2. Enlarging face areas
enlarge the face bounding box by 30%

5.4. Shape initialization
Most face alignment methods start from a rough initialization, and then refine the shape iteratively until convergence. The initialization step typically has great influence on the final result, and an initial shape far from the ground truth might lead to very bad alignment results.

5.5. Accuracy and efficiency tradeoffs
Face alignment in real time is crucial to many practical applications. The efficiency mainly depends on the feature extraction and shape prediction steps

6 System Evaluation
测试评估使用的数据库有：

11