[reading notes] Constrained Local Neural Fields for robust facial landmark detection in the wild

CLNF  in  OpenFace    (upgrade version of CLM framework) 

Constrained Local Neural Fields for robust facial landmark detection in the wild by Tadas Baltrusaitis, Peter Robinson, and Louis-Philippe Morency. [2013]

1. flow chart and innovation of this paper

2. traditional Constrained Local Model(CLM)

Our approach uses the Constrained Local Model (CLM) framework.

 There are three main parts to a CLM: 

Part A: a point distribution model (PDM),

           which is parameter p in formula (1). 

Part B: patch experts, 

            which is used to calculate xi in formula (1).

            Refer to section 2.2.1 Patch Expert for detail.(ToDo)

Part C: the fitting approach, 

            which is used to find optimized p* in formula (1).

            Refer to section 2.2.2 Regularised Landmark Mean Shift for detail.(ToDo)

3. Constrained Local Neural Field (CLNF) landmark detection

difference between traditional CLM and the algorithm proposed in this paper CLNF:

Part A: a point distribution model (PDM),

           remain the same as traditional  CLM. 

Part B: patch experts, 

           proposed CLNF includes a novel Local Neural Field patch expert,

        which learns the non-linearities and spatial relationships between pixel values and the probability of landmark alignment.

            Figure 3 is the schematic diagram of this novel patch expert.

            Figure 2 is the experiment results of traditional patch expert in CLM and this novel patch expert in CLNF.

            Refer to 3.1. Local Neural Field patch expert for detail.(ToDo)

Part C: the fitting approach, 

           proposed CLNF also uses a novel Non-uniform Regularised Landmark Mean Shift fitting technique,

           which takes patch reliabilities into consideration.

           Figure 4a is the reliabilities of CLNF patch experts used in one experiment.

           Refer to section 3.2. Non-uniform RLMS for detail.(ToDo)

4. Experiment Result

(a) Cumulative error curves

out-performing all of the other baselines tested when performing landmark detection in unseen lighting conditions and in the wild.

(b) Computational complexity

CLNF is also a fast approach: a Matlab implementation can process 2 images per second on in the wild data.

but still much more slower than the following approach.

1. One Millisecond Face Alignment with an Ensemble of Regression Trees by Vahid Kazemi and Josephine Sullivan. [2014]