SDM For Face Alignment 流程介绍及Matlab代码实现之预处理篇

SDM全称为 Supervised Descent Method，是一种机器学习的方法，可以被用来做Face Alignment.
下面我们将通过matlab代码来梳理整个实现的过程。

预处理阶段

Input： ../data/lfpw/trainset （811张图片）
Output: mean_shape 811张图片的特征点的平均值

我们从网上download下训练数据集,包括image和ground-truth points， 我们希望可以得到所有图片的平均特征点，但是由于每张图片的尺寸各异，图片里的人脸也是各不相同，因此，只是简简单单将ground-truth points平均一下是没有意义的，所以必须把他们统一到一个尺寸下。

我们可以提取人脸，将其放缩到400*400的尺寸下。然后通过取变换后的特征点的平均值来作为平均特征点。那么如何进行呢？方法如下：

先正则化第一张图片

1.取第一张图片ground-truth points的包围盒（即包含特征点的最小矩形）。
2.将包围盒的左上角向坐标系左上角平移包围盒一半的宽和高，作为新的包围盒的左上角，宽和高分别取原来的2倍。这样裁剪出的人脸就基本上是人的正脸了，同时相应的变换特征点的位置。
3.放缩上面新得到的图片到400*400，同时相应的变换特征点的位置。
这样第一张图片的400*400的正脸以及相应的特征点就取得了。
如下图：

再正则其他图片

我们通过普氏分析将其他图片的特征点与第一张正则化的特征点对齐，获得统一尺寸下的特征点，这样就可求解平均值了。
bounding_box.m代码，用来裁剪正脸:

function [cropmin,cropmax,offset,minshape,marginW,marginH] = ...
    bounding_box ( shape,img )
%cropmin,cropmax分别是由特征点的包围盒延拓的正脸的左上角和右下角
% if(offset==[0 0]表示正脸未跃出图片
% else 人脸需要做平移，平移后的左上角的坐标点为（1,1），平移的长度为offset
% minshape:特征点包围盒的左上角
% marginW：特征点包围盒的宽的一半
% marginH：特征点包围盒的高的一半
%shape:特征点，以水平的为x，以竖直的为y,同matlab的图像处理工具箱的相反
minshape = min(shape);%min_x,min_y
maxshape = max(shape);%max_x,max_y

%% calculating bounding box %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
width   = maxshape(1) - minshape(1);
height  = maxshape(2) - minshape(2);

marginW = width/2;
marginH = height/2;

cropmin = round(minshape - [marginW marginH]);
cropmax = round(maxshape + [marginW marginH]);


SIZE= size(img);%由于是彩色图，所以SIZE是三维，因此不能写成[m,n]

offset = [0 0];%前面的盒子求出了正脸的大小包围盒，但是如果一张照片中的头像偏向左边和上边，将导致求出的正脸包围盒超过原点，越出图像，因此需要将正脸平移，平移的尺寸为offset()+1，平移后的正脸左上角坐标为（1,1）

if(cropmin(1)<=0)
    offset(1)  = -cropmin(1);
    cropmin(1) = 1;
end
if(cropmin(2)<=0)
    offset(2)  = -cropmin(2);
    cropmin(2) = 1;
end
% %如下为补充项,防止裁剪的图片过大超过原图片的边界
if(cropmax(1)>=SIZE(2))
    cropmax(1) = SIZE(2);
end

if(cropmax(2)>=SIZE(1))
    cropmax(2) = SIZE(1);
end
end

normalize_first_shape.m代码：处理第一张图片

function [shape] = normalize_first_shape( Data, options )

shape = Data.shape;
image = Data.img;%补充项
image=imread(image);
%% calculating bounding box %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
[cropmin,cropmax,offset,minshape,marginW,marginH] = bounding_box ( shape,image );

%%输出offset不为0的图片的位置
%{
 if offset~=[0 0]
disp('我们要找的头像偏左或偏上的图片已找到,地址为：');
disp(Data.img);
pause;
end
%}


%% calculate scale factor %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
W_H = cropmax - cropmin;
wh1 = W_H(1);
wh2 = W_H(2);

CanvasSize = options.canvasSize;%标准的正脸大小

scf = CanvasSize(1)/wh1;
if(scf*wh2 > CanvasSize(2))
    scf = CanvasSize(2)/wh2;
end

%% croping image (for debug only) %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
debug =0;

if debug 
    img = imread(Data.img);
    cropImage   = img(cropmin(2):cropmax(2), cropmin(1):cropmax(1));
    scaleImage  = imresize(cropImage, scf);
end


%% scale shape and image %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

shape = shape - repmat((minshape - [marginW marginH] + offset) ...
    , size(shape, 1), 1);  
shape = shape*scf;

if debug
    % Displaying image and feature points.
    figure(1);
    imshow(image);
    figure(3);
    imshow(scaleImage);
    hold on;
    plot(shape(:, 1), shape(:, 2), 'g*');
    pause;
end

end

normalize_rest_shape.m：依据正则化的第一张图片特征点来正则化其他图片的特征点。

function [shape,img] = normalize_rest_shape ( ref, data, options )

cvw = options.canvasSize(1);
cvh = options.canvasSize(2);

base = ref.shape;

shape = data.shape;

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Use procrustes analysis to align shape.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
[d, z, tform] = procrustes(base, shape, 'Reflection',false);

%% normaling shape %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
debug =0;

if debug

    Trans = -1/tform.b*tform.c*tform.T';
    Trans = Trans(1, :);

    transM = [1/tform.b*tform.T Trans'];
    cvXY   = [1 cvw 1 cvw;
        1 1 cvh cvh];

    img       = im2double(rgb2gray(imread(data.img)));
    normImg   = quad2Box(img, cvXY, transM);

    figure(2);
    imshow(normImg);
    hold on;
    plot(z(:, 1), z(:, 2), 'r.');
    pause;

end

shape = z;

end

然后求解平均特征点即可。