SDM人脸对齐系列一：数据预处理

    人脸对齐是人脸识别系统中很重要的一个环节，SDM是传统人脸对齐算法中性能较为不错的一种，在今天这个深度学习如火如荼的时代，SDM依旧具有一定的优势。SDM相比深度网络具有模型小，速度快等优点。尽管SDM已经出现了好几年，但是网络上对其具体的详细讲解的知识还是比较少，尤其是和训练相关的东西。这几天自己硬着头皮啃了下源码，给出自己的理解。

1、数据集的下载

    关于人脸对对齐的数据集有很多，比较常用的几个库可以到这里下载：点击打开链接。本文采用的是lfw数据集进行的实验，下载后可以看到数据集分为testset和trainset两个文件夹,分别包含了测试集和训练集，各个数据集下包含了图片和对应的标签。

2、求平均人脸形状

    人脸对齐中，较为关键的第一步就是获取训练集的人脸关键点的平均形状。因为测试数据集是基于平均脸进行偏移实现人脸对齐的。首先，正则化第一张人脸图片：

(1)取第一张图片包含gt points的最小矩形。

(2)以得到的最小矩形向左上角平移，平移的大小x、y方向分别为矩形宽和高，然后矩形的宽和高分别扩展为原来的两倍，将这个变化后的矩形对图片中的人脸进行裁剪，同时对裁剪出来的人脸的特征点做相应的调整，这样裁剪出来的人脸基本上包含了整个人脸部分。

(3)将新裁剪出来的人脸缩放到400*400的大小，同时对应的特征点做相应的调整，当然这里你不一定也要设置为400*400，可以根据自己的需要修改。这里只是讲解下主要涉及的函数，太具体的需要大家自己摸索，具体代码如下：

bounding_box.m用来获取人脸裁剪的信息：

function [cropmin,cropmax,offset,minshape,marginW,marginH] = ...
    bounding_box ( shape )

%获取特征点的人脸包围盒
minshape = min(shape);
maxshape = max(shape);

%% calculating bounding box %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
width   = maxshape(1) - minshape(1);
height  = maxshape(2) - minshape(2);

marginW = width/2;
marginH = height/2;

cropmin = round(minshape - [marginW marginH]);%最左上角的点
cropmax = round(maxshape + [marginW marginH]);%最右下角的点

offset = [0 0];

if(cropmin(1)<=0)
    offset(1)  = -cropmin(1);
    cropmin(1) = 1;
end

if(cropmin(2)<=0)
    offset(2)  = -cropmin(2);
    cropmin(2) = 1;
end

end

normalize_first_shape.m用于正则化第一张图片

function [shape] = normalize_first_shape( Data, options )

shape = Data.shape;

%% calculating bounding box %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
[cropmin,cropmax,offset,minshape,marginW,marginH] = bounding_box ( shape );%获取原始图片裁剪信息

%% calculate scale factor %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
W_H = cropmax - cropmin;
wh1 = W_H(1);
wh2 = W_H(2);

CanvasSize = options.canvasSize;

scf = CanvasSize(1)/wh1;
if(scf*wh2 > CanvasSize(2))
    scf = CanvasSize(2)/wh2;
end

%% croping image (for debug only) %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
debug = 0;

if debug 
    img = imread(Data.img);
    cropImage   = img(cropmin(2):cropmax(2), cropmin(1):cropmax(1));
    scaleImage  = imresize(cropImage, scf);
end


%% scale shape and image %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%调整对应的人脸特征点
shape = shape - repmat((minshape - [marginW marginH] + offset) ...
    , size(shape, 1), 1);
shape = shape*scf;

if debug
    % Displaying image and feature points.
    figure(1);
    imshow(scaleImage);
    hold on;
    plot(shape(:, 1), shape(:, 2), 'g*');
    pause;
end

end

利用第一张正则化特征点来正则化其他图片的特征点

function [shape,img] = normalize_rest_shape ( ref, data, options )

cvw = options.canvasSize(1);
cvh = options.canvasSize(2);

base = ref.shape;

shape = data.shape;

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Use procrustes analysis to align shape.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%利用普氏分析法将其他图片与第一张图片对齐，从而获得平均形状
[d, z, tform] = procrustes(base, shape, 'Reflection',false);

%% normaling shape %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
debug = 0;

if debug
    
    Trans = -1/tform.b*tform.c*tform.T';
    Trans = Trans(1, :);
    
    transM = [1/tform.b*tform.T Trans'];
    cvXY   = [1 cvw 1 cvw;
        1 1 cvh cvh];
    
    img       = im2double(rgb2gray(imread(data.img)));
    normImg   = quad2Box(img, cvXY, transM);
    
    figure(1);
    imshow(normImg);
    hold on;
    plot(z(:, 1), z(:, 2), 'r.');
    pause;
    
end

shape = z;

end

顶层归一化函数

function Data = normalize_data ( Data, options )

n = length(Data);

%% noramlizing the first image
%正则化第一张图片
shape          = normalize_first_shape( Data(1) , options );
Data(1).shape  = shape;


%% using the first to noramlizing others.
%根据第一张图片正则化第二张人脸图片
for i = 2 : n
    [shape] = normalize_rest_shape( Data(1), Data(i), options );
    Data(i).shape  = shape;
end

end

形状学习，获取平均形状

function do_learn_shape ( options )

%学习人脸的平均形状

%% locating data folders %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
imageDataPath = options.trainingImageDataPath;%训练数据的路径
truthDataPath = options.trainingTruthDataPath;%训练数据的标签

%% loading training data %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Data = load_data( imageDataPath, truthDataPath, options );%加载图片和标签

%% normalizing training data %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Data = normalize_data( Data, options );%归一化人脸形状

%% shape model training %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
ShapeModel = build_shape_model( Data );%对归一化后的人脸特征点求平均形状

if ~exist( options.modelPath , 'dir' )
    mkdir( options.modelPath );
end

%保存人脸的平均形状
save([options.modelPath options.slash options.datasetName ...
    '_ShapeModel.mat'], 'ShapeModel');

clear Data;
clear ShapeModel;

下面的效果图是通过主成分分析归一化后的人脸形状获得的平均形状

若有不当之处，请指教