Paper Reading: Pose-Aware Face Recognition in the wild

Pose-Aware Face Recognition in the wild (CVPR 2016)

paper link: https://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Masi_Pose-Aware_Face_Recognition_CVPR_2016_paper.pdf

Abstract

• proposed a method that explicitly tackles pose variations by using multiple pose-specific models and rendered face images.
• Dataset used:
  – CASIA
  – IJB-A
  – PIPA

Proposed Method

• assumes that in general the face pose distribution p(p|I), given one image I, is not dominated by near-frontal faces. As such, the author proposed a method to learn multiple pose-specific CNN models, assuming detected landmarks on an image.

• compensating for out-of-plane variations using frontalization can be a noisy process
• As such, the author proposed a method that considers different ways of aligning a face.

• 2D in-plane alignment

• 3D out-of-plane alignment:

  render images at a specific yaw value. This paper uses unmodified 3D generic face mode which not only use frontalization but also render images to half-profile (40 degree) and full profile (75 degree) views in order to cope for extreme yaw variations

Pose-Aware CNN Models (PAM)

treat each type of alignment and data independently. i.e. To learn and specific model for each type of alignment and mode of pose distribution.

• Discovering the training pose distribution

  – From detected landmarks in CASIA, the author estimate the pose of the face putting in corresponding 2D detected landmarks with 3D labeled landmarks on 3D generic model M. Then a perspective camera model mapping the generic 3D model M on the image can be estimated.
  – PnP method is used to estimate external camera parameters.