This note is a brief summary of the 3DMM paper A Morphable Model For The Synthesis Of 3D Faces.
Note: I have no idea why there is a “|” following each math environment and “\bold” has no effect, for clarity, see the original version 3D Morphable Model Method.
- 3D head lase scans
- full correspondence of faces (the method for acquiring this condition is described in the last section of the note.)
The model construction process consists of two steps: compute correspondence and construct model. Notice that these are steps for TRAINING, when the model is constructed, we can apply this model to new faces and scans through matching algorithm.
concept of morphable face model
A face has two major properties: geometry represented as shape-vector
Notice that this representation is based on exemplar faces, we actually use the PCA form in next section to perform reconstruction.
The construction process can be described as a PCA procedure, i.e., use principle components(eigenvectors of convariance matrices of shape and texture) to represent the model:
To quantify the results in terms of the plausibility of being faces, the author fits a multivariate normal distribution to the data set of 200 faces, then the probability for coefficients
In addition, we can divide the faces into independent subregions that are morphed independently. In this paper, the author defines four subregions, by which the complete 3D face is generated by computing linear combinations for each segment simultaneously and blending them at the borders according to algorithm .
To map facial attributes(gender, fullness of faces, darkness of eyebrows, double chins, hooked and concave noses) defined by hand-labeled set of example faces to the parameter space of the morphable model, first define shape and texture vectors that will manipulate a specific attribute:
Application– Matching 3DMMs to images and 3D scans
Matching a morphable model to images is to optimize the coefficients of the 3D model(
are rendered using perspective projection and the Phong illumination model. To estimate the maximum posterior probability of
The optimazation of this cost function is the process of obtain the optimal parameters
If we neglect correlations between some of the variables, the right-hand side is
In the process of optimization, we need to inference
The above are the procedure to map a 3D morphable model to images, in order to apply to scans, we just need to replace
Building morphable model without correspondence
All process stated above are based on the assumption that all exemplar faces are in full correspondence. This section will describe two algorithms for computing correspondence.
3D corresponding using optic flow
Optic flow is first proposed to estimate corresponding points in images
Bootstrapping the model
Since optic flow does not incorporate any constraints on the set of solutions, it fails on some of the more unusual faces in the database. The modified bootstrapping method improve correspondence iteratively.
The process if as follows:
1. use optic flow to compute preliminary correspondences between faces and a reference face.
2. compute morphable models based on the correspondences and average faces as new reference face.
3. match the models to 3D scans, now we have original scans and approximated scans.
4. compute the correspondences between the two scans using optic flow.
5. iterate above steps.
 P.J. Burt and E.H. Adelson. Merging images through pattern decomposition. In Applications of Digital Image Processing VIII, number 575, pages 173–181. SPIE The International Society for Optical Engeneering, 1985.
 Blanz,V.,&Vetter,T.(2003).Face recognition based on fitting a 3d morphable model. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 25(9), 1063–1074.
 T. Vetter and V. Blanz. Estimating coloured 3d face models from single images:An example based approach. In Burkhardt and Neumann, editors, ComputerVision – ECCV’98 Vol. II, Freiburg, Germany, 1998. Springer, Lecture Notes in Computer Science 1407.