face_alignment库获取人脸图片landmark示例

前言

face_alignment是Python环境下一个用于检测人脸关键点（landmark）的常用的库。基于FAN方法实现，具有2D和3D关键点检测的能力。

方法示例

face-alignment库获取输入图片中的人脸关键点，图片输入格式为RGB通道顺序格式，因此如果用OpenCV读取图片后需要注意转换通道顺序，如果是用skimage.io读取则不用转换通道顺序。

2D test on CPU

import cv2
import face_alignment

def cv_draw_landmark(img_ori, pts, box=None, color=(0, 0, 255), size=2):
    img = img_ori.copy()
    n = pts.shape[0]
    for i in range(n):
        cv2.circle(img, (int(round(pts[i, 0])), int(round(pts[i, 1]))), size, color, -1)
        # cv2.putText(img, str(i), (int(round(pts[i, 0])-3), int(round(pts[i, 1])-3)), cv2.FONT_HERSHEY_COMPLEX, 0.3, (0,255,0), 1)
    return img


if __name__ == "__main__":
	img_path = 'test.png'
	fa = face_alignment.FaceAlignment(face_alignment.LandmarksType._2D, flip_input=False, device='cpu')
	input = cv2.imread(img_path)
	preds = fa.get_landmarks(input[..., ::-1])  # 翻转BGR为RGB
	print('landmark68 2D preds: ', preds)
	output = cv_draw_landmark(input, preds[0])
	cv2.imshow('result', output)
	cv2.waitKey(0)
	

3D test on GPU

import face_alignment
from skimage import io

if __name__ == "__main__":
	img_path = 'test.png'
	fa = face_alignment.FaceAlignment(face_alignment.LandmarksType._3D, flip_input=False, device='cuda:0')
	input = io.imread(img_path)
	preds = fa.get_landmarks(input)
	print('landmark68 3D preds: ', preds)
	

源码简析

（1） get_landmarks方法已弃用，可改为调用get_landmarks_from_image方法。

（2）get_landmarks_from_image方法只传图片的情况下会先检测人脸，然后再检测关键点。

如果不需要使用face_alignment库的人脸检测，可以在调用get_landmarks_from_image方法时传入人脸bbox。

fa = face_alignment.FaceAlignment(face_alignment.LandmarksType._2D, flip_input=False, device='cpu')
input = cv2.imread(path)
bbox = [[0,0,255,255]] # 传入的人脸bbox
preds = fa.get_landmarks(input[..., ::-1], bbox)
print(preds)

关键点图示

2D关键点位置

2D关键点编号

使用face_alignment库提取人脸嘴部区域

import os
from skimage import io
from skimage.transform import resize
import face_alignment

# use face_alignment to extract mouth area
fa = face_alignment.FaceAlignment(face_alignment.LandmarksType._2D, flip_input=False, device='cuda:0')
img_path = 'test.png'
img = io.imread(img_path)

bbox = [[0,0,255,255]] # bbox for 256*256 img
preds = fa.get_landmarks_from_image(img, bbox)
mouth = np.zeros((64, 64, 3), dtype='float32')
mouth_center = np.zeros((2), dtype='int')

if preds is not None:
    if len(preds) > 0:
        ldmk68 = preds[0]
        mouth_center = np.mean(ldmk68[48:], axis=0)  # 48-67: mouth
        mouth_center = mouth_center.astype(np.int)
        pt1 = tuple(mouth_center - 32)
        pt2 = tuple(mouth_center + 32)
        mouth = img[pt1[1]:pt2[1],pt1[0]:pt2[0]]
        if mouth.shape[0]!=0 and mouth.shape[1]!=0:
            if mouth.shape[0] != 64 or mouth.shape[1] != 64:
                mouth = resize(mouth, (64, 64), anti_aliasing=True)
        else:
            mouth = np.zeros((64, 64, 3), dtype='float32')
            mouth_center = np.zeros((2), dtype='int')
mouth = mouth.transpose((2, 0, 1))

参考资料

[1] GitHub - 1adrianb / face-alignment
[2] get_landmarks_from_image方法