Face Recognition的API源码阅读、识别流程了解

Face Recognition 人脸识别Git地址

开源项目Git地址

• 这是一个基于Dlib的人脸识别库，它给我的第一感觉就是——简洁明了。
• 根据官方文档的中文版所述：本项目face_recognition是一个强大、简单、易上手的人脸识别开源项目，并且配备了完整的开发文档和应用案例，特别是兼容树莓派系统。我经过这几天的使用，发现确实如此。

今天是来阅读这个库的API的

• 这里是API的源码地址
• 阅读源码，一个比较简单的方法是用官方示例去debug，下面给出一段官方示例

识别关键点

import face_recognition
image = face_recognition.load_image_file("your_file.jpg")
face_landmarks_list = face_recognition.face_landmarks(image)

• 三行代码确实有够简洁的，而且我们可以根据函数名很容易明白这个函数是干嘛的；

1.1.face_recognition.load_image_file

• 我们在API文件中搜索函数名即可找到对应的函数源码

def load_image_file(file, mode='RGB'):
    """
    Loads an image file (.jpg, .png, etc) into a numpy array
    :param file: image file name or file object to load
    :param mode: format to convert the image to. Only 'RGB' (8-bit RGB, 3 channels) and 'L' (black and white) are supported.
    :return: image contents as numpy array
    """
    im = PIL.Image.open(file)
    if mode:
        im = im.convert(mode)
    return np.array(im)

• 关键信息有：Loads an image file (.jpg, .png, etc) into a numpy array，PIL，np，
• 连起来就是用PIL吧图片加载成numpy数组
• 这意味着两点：PIL、numpy：
  • 它用的PIL读，也就是说我们可以用其他读取方式，比如OpenCV之类，只要返回np.array即可

1.2.face_recognition.face_landmarks

• 同样的方法搜索face_landmarks我们回搜到两个：face_landmarks和_raw_face_landmarks，我们先说前者，face_landmarks源码如下所示

def face_landmarks(face_image, face_locations=None, model="large"):
    """
    Given an image, returns a dict of face feature locations (eyes, nose, etc) for each face in the image
    :param face_image: image to search
    :param face_locations: Optionally provide a list of face locations to check.
    :param model: Optional - which model to use. "large" (default) or "small" which only returns 5 points but is faster.
    :return: A list of dicts of face feature locations (eyes, nose, etc)
    """
    landmarks = _raw_face_landmarks(face_image, face_locations, model)
    landmarks_as_tuples = [[(p.x, p.y) for p in landmark.parts()] for landmark in landmarks]

    # For a definition of each point index, see https://cdn-images-1.medium.com/max/1600/1*AbEg31EgkbXSQehuNJBlWg.png
    if model == 'large':
        return [{
            "chin": points[0:17],
            "left_eyebrow": points[17:22],
            "right_eyebrow": points[22:27],
            "nose_bridge": points[27:31],
            "nose_tip": points[31:36],
            "left_eye": points[36:42],
            "right_eye": points[42:48],
            "top_lip": points[48:55] + [points[64]] + [points[63]] + [points[62]] + [points[61]] + [points[60]],
            "bottom_lip": points[54:60] + [points[48]] + [points[60]] + [points[67]] + [points[66]] + [points[65]] + [points[64]]
        } for points in landmarks_as_tuples]
    elif model == 'small':
        return [{
            "nose_tip": [points[4]],
            "left_eye": points[2:4],
            "right_eye": points[0:2],
        } for points in landmarks_as_tuples]
    else:
        raise ValueError("Invalid landmarks model type. Supported models are ['small', 'large'].")

• 其源码简洁明了：
  • 先看返回值：我们知道模型有大小之分，区别就是关键点数不同，return的内容是map，<String,list>这样子，指明了关键点的位置，
  • 除此之外还有两句,分别为:
    • landmarks = _raw_face_landmarks(face_image, face_locations, model)
    • landmarks_as_tuples = [[(p.x, p.y) for p in landmark.parts()] for landmark in landmarks]
    • landmarks_as_tuples 这句就是化成元组，简单理解就是数据整形
    • 看到landmarks 这句是我们发现调用了_raw_face_landmarks方法，也就是说_raw_face_landmarks才是执行代码，而face_landmarks不过是结果处理。下面就来看_raw_face_landmarks

1.3 _raw_face_landmarks

def _raw_face_landmarks(face_image, face_locations=None, model="large"):
    if face_locations is None:
        face_locations = _raw_face_locations(face_image)
    else:
        face_locations = [_css_to_rect(face_location) for face_location in face_locations]

    pose_predictor = pose_predictor_68_point

    if model == "small":
        pose_predictor = pose_predictor_5_point

    return [pose_predictor(face_image, face_location) for face_location in face_locations]

• 我们一句一句来分析：
  • face_locations 指的是人脸坐标，也就是框的位置，默认为None，实际上调用的是_raw_face_locations函数,它里面根据模型类型调用:
    • face_detector = dlib.get_frontal_face_detector()或者
    • cnn_face_detector = dlib.cnn_face_detection_model_v1(cnn_face_detection_model)
  • 这里就涉及到Dlib的API的，在此不再展开，此处的目的就是识别人脸得到人脸坐标，这意味着，我们可以用自己的目标识别算法来识别人脸位置，然后将人脸位置穿进去，也就是else里面的内容，那就是传入自定义人脸位置，用的列表表达式，然后用_css_to_rect方法调用dlib.rectangle方法变成特定形式。
  • 接下来是加载：pose_predictor,这个是根据模型的类型加载68_point还是5_point的关键点提取器（这个是Dlib的接口）。
  • 最后就是遍历人脸位置数组，过一遍pose_predictor，得到关键点信息就可以返回了。
  • 最后结果借用一下官方的图，另外官方还给出了画关键点的代码~
  • 

人脸识别

• 这是第二段示例代码：

import face_recognition
known_image = face_recognition.load_image_file("biden.jpg")
unknown_image = face_recognition.load_image_file("unknown.jpg")

biden_encoding = face_recognition.face_encodings(known_image)[0]
unknown_encoding = face_recognition.face_encodings(unknown_image)[0]

results = face_recognition.compare_faces([biden_encoding], unknown_encoding)

• 这段代码也比较简单明了，load_image_file上网说过了，直接看face_encodings的源码：

2.1.face_recognition.face_encodings

def face_encodings(face_image, known_face_locations=None, num_jitters=1, model="small"):
    """
    Given an image, return the 128-dimension face encoding for each face in the image.
    :param face_image: The image that contains one or more faces
    :param known_face_locations: Optional - the bounding boxes of each face if you already know them.
    :param num_jitters: How many times to re-sample the face when calculating encoding. Higher is more accurate, but slower (i.e. 100 is 100x slower)
    :param model: Optional - which model to use. "large" (default) or "small" which only returns 5 points but is faster.
    :return: A list of 128-dimensional face encodings (one for each face in the image)
    """
    raw_landmarks = _raw_face_landmarks(face_image, known_face_locations, model)
    return [np.array(face_encoder.compute_face_descriptor(face_image, raw_landmark_set, num_jitters)) for raw_landmark_set in raw_landmarks]

• 乍一看这么长，实际上都是参数说明，别的都一目了然，特别提一句就是num_jitters是值特征提取次数
• 第一句关键点提取：_raw_face_landmarks,参考上文
• 第二句特征提取：face_encoder.compute_face_descriptor,直接搜索compute_face_descriptor发现没有，再搜索face_encoder,是一句Dlib的接口调用
  • face_encoder = dlib.face_recognition_model_v1(face_recognition_model)
  • 接着face_recognition_model是由它的上一句产生：face_recognition_model = face_recognition_models.face_recognition_model_location()
• 再追下去就是一段import了，也就是需要导入人脸识别模型，或者说特征提取模型

try:
    import face_recognition_models
except Exception:
    print("Please install `face_recognition_models` with this command before using `face_recognition`:\n")
    print("pip install git+https://github.com/ageitgey/face_recognition_models")
    quit()

• 提取完了就转成np数组返回；

2.2.face_recognition.compare_faces

def compare_faces(known_face_encodings, face_encoding_to_check, tolerance=0.6):
    """
    Compare a list of face encodings against a candidate encoding to see if they match.
    :param known_face_encodings: A list of known face encodings
    :param face_encoding_to_check: A single face encoding to compare against the list
    :param tolerance: How much distance between faces to consider it a match. Lower is more strict. 0.6 is typical best performance.
    :return: A list of True/False values indicating which known_face_encodings match the face encoding to check
    """
    return list(face_distance(known_face_encodings, face_encoding_to_check) <= tolerance)

• compare_faces调用的是face_distance

2.3.face_distance

def face_distance(face_encodings, face_to_compare):
    """
    Given a list of face encodings, compare them to a known face encoding and get a euclidean distance
    for each comparison face. The distance tells you how similar the faces are.
    :param faces: List of face encodings to compare
    :param face_to_compare: A face encoding to compare against
    :return: A numpy ndarray with the distance for each face in the same order as the 'faces' array
    """
    if len(face_encodings) == 0:
        return np.empty((0))

    return np.linalg.norm(face_encodings - face_to_compare, axis=1)

• 而face_distance则是简单的人脸值比对，返回二者的差值，再在compare_faces中与tolerance，即阈值比较，如果小于此值，则说明识别成功，那就返回对于的人名，人脸识别就此就结束了。
• 官方的效果图是：