【人脸识别】旷世的好用的人脸识别 insightface 使用记录和搭建服务注意点和坑，手把手教你从0到1，太好用了，可以直接运行项目，跟进更新+ffmpeg拉流抽帧

---

前言

人脸识别项目，再走一遍。之前是公司老人留下的，没部署过，没交付过。这次重新搞一个，并且把搞过的记录下来。

---

参考旷视的insightface:
https://github.com/deepinsight/insightface/tree/master/python-package
另外一个blog:https://blog.csdn.net/u013171226/article/details/123249453

创建环境，安装库

conda activate -n insightface python=3.8
pip install insightface

把它的git 代码都拉下来：
https://github.com/deepinsight/insightface
解压到 /home/jianming_ge/workplace/zhongwaiyun/insightface/

1.开始

1.1 前置

cd /home/jianming_ge/workplace/zhongwaiyun/insightface/python-package
pip install numpy opencv-python 
pip install onnxruntime-gpu

创建一个文件：
aaa.py

import cv2
import numpy as np
import insightface
from insightface.app import FaceAnalysis
from insightface.data import get_image as ins_get_image

app = FaceAnalysis(allowed_modules=['detection'],providers=['CUDAExecutionProvider', 'CPUExecutionProvider'],download=False)
app.prepare(ctx_id=0, det_size=(640, 640))
img = ins_get_image('ldh')  #不用带后缀，图片放到./insightface/python-package/insightface/data/images
faces = app.get(img)
print("faces::::", faces)
rimg = app.draw_on(img, faces)
cv2.imwrite("./ldh_output.jpg", rimg)

运行:python aaa.py
报错：

(insightface) [jianming_ge@localhost python-package]$ python aaa.py 
Traceback (most recent call last):
  File "aaa.py", line 3, in <module>
    import insightface
  File "/home/jianming_ge/workplace/zhongwaiyun/insightface/python-package/insightface/__init__.py", line 18, in <module>
    from . import app
  File "/home/jianming_ge/workplace/zhongwaiyun/insightface/python-package/insightface/app/__init__.py", line 2, in <module>
    from .mask_renderer import *
  File "/home/jianming_ge/workplace/zhongwaiyun/insightface/python-package/insightface/app/mask_renderer.py", line 8, in <module>
    from ..thirdparty import face3d
  File "/home/jianming_ge/workplace/zhongwaiyun/insightface/python-package/insightface/thirdparty/face3d/__init__.py", line 3, in <module>
    from . import mesh
  File "/home/jianming_ge/workplace/zhongwaiyun/insightface/python-package/insightface/thirdparty/face3d/mesh/__init__.py", line 9, in <module>
    from .cython import mesh_core_cython

解决：

 find  ./ -iname "cython"
 (insightface) [jianming_ge@localhost python-package]$ find  ./ -iname "cython"
./insightface/thirdparty/face3d/mesh/cython
(insightface) [jianming_ge@localhost python-package]$ cd ./insightface/thirdparty/face3d/mesh/cython/
(insightface) [jianming_ge@localhost cython]$ ls
mesh_core.cpp  mesh_core_cython.c  mesh_core_cython.cpp  mesh_core_cython.pyx  mesh_core.h  setup.py
(insightface) [jianming_ge@localhost python-package]$ cd ./insightface/thirdparty/face3d/mesh/cython/
(insightface) [jianming_ge@localhost cython]$ ls
mesh_core.cpp  mesh_core_cython.c  mesh_core_cython.cpp  mesh_core_cython.pyx  mesh_core.h  setup.py
(insightface) [jianming_ge@localhost cython]$ python setup.py build_ext -i
In file included from /home/jianming_ge/miniconda3/envs/insightface/lib/python3.8/site-packages/numpy/core/include/numpy/ndarraytypes.h:1940,
                 from /home/jianming_ge/miniconda3/envs/insightface/lib/python3.8/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
                 from /home/jianming_ge/miniconda3/envs/insightface/lib/python3.8/site-packages/numpy/core/include/numpy/arrayobject.h:5,
                 from mesh_core_cython.cpp:780:
/home/jianming_ge/miniconda3/envs/insightface/lib/python3.8/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
 #warning "Using deprecated NumPy API, disable it with " \

到包文件下，执行一下安装就可以了
cd ./insightface/thirdparty/face3d/mesh/cython/
python setup.py build_ext -i

1.2 再次运行，人脸检测跑通

python aaa.py

(insightface) [jianming_ge@localhost python-package]$ python aaa.py 
download_path: /home/jianming_ge/.insightface/models/buffalo_l
Downloading /home/jianming_ge/.insightface/models/buffalo_l.zip from https://github.com/deepinsight/insightface/releases/download/v0.7/buffalo_l.zip...
  0%|▏                                                                                                                                                                                     | 208/281857 [00:01<43:44, 107.30KB/s]^Z
[2]+  已停止               python aaa.py

他去下载去了，不知为什么在从git上下载特别慢，所以放弃了，想法是手动下载好，传到它所需要的目录。
Downloading /home/jianming_ge/.insightface/models/buffalo_l.zip from https://github.com/deepinsight/insightface/releases/download/v0.7/buffalo_l.zip…
代码追踪后在这里，dir_path

位置:python-package/insightface/utils/storage.py
追踪到dir_path = ~/.insightface/models/buffalo_l
建好目录，并且把权重文件放进去

(insightface) [jianming_ge@localhost python-package]$ mkdir ~/.insightface/models/buffalo_l
(insightface) [jianming_ge@localhost python-package]$ cp ~/.insightface/models/*.onnx  ~/.insightface/models/buffalo_l/
(insightface) [jianming_ge@localhost python-package]$ ls ~/.insightface/models/buffalo_l
1k3d68.onnx  2d106det.onnx  det_10g.onnx  genderage.onnx  w600k_r50.onnx

import os
import os.path as osp
import zipfile
from .download import download_file

BASE_REPO_URL = 'https://github.com/deepinsight/insightface/releases/download/v0.7'

def download(sub_dir, name, force=False, root='~/.insightface'):
    _root = os.path.expanduser(root)
    dir_path = os.path.join(_root, sub_dir, name)
    if osp.exists(dir_path) and not force:
        return dir_path
    print('download_path:', dir_path)
    zip_file_path = os.path.join(_root, sub_dir, name + '.zip')
    model_url = "%s/%s.zip"%(BASE_REPO_URL, name)
    download_file(model_url,
             path=zip_file_path,
             overwrite=True)
    if not os.path.exists(dir_path):
        os.makedirs(dir_path)
    with zipfile.ZipFile(zip_file_path) as zf:
        zf.extractall(dir_path)
    #os.remove(zip_file_path)
    return dir_path

def ensure_available(sub_dir, name, root='~/.insightface'):
    return download(sub_dir, name, force=False, root=root)

def download_onnx(sub_dir, model_file, force=False, root='~/.insightface', download_zip=False):
    _root = os.path.expanduser(root)
    model_root = osp.join(_root, sub_dir)
    new_model_file = osp.join(model_root, model_file)
    if osp.exists(new_model_file) and not force:
        return new_model_file
    if not osp.exists(model_root):
        os.makedirs(model_root)
    print('download_path:', new_model_file)
    if not download_zip:
        model_url = "%s/%s"%(BASE_REPO_URL, model_file)
        download_file(model_url,
                 path=new_model_file,
                 overwrite=True)
    else:
        model_url = "%s/%s.zip"%(BASE_REPO_URL, model_file)
        zip_file_path = new_model_file+".zip"
        download_file(model_url,
                 path=zip_file_path,
                 overwrite=True)
        with zipfile.ZipFile(zip_file_path) as zf:
            zf.extractall(model_root)
        return new_model_file

再次执行：
python aaa.py
输出：

(insightface) [jianming_ge@localhost python-package]$ python aaa.py 
Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}, 'CUDAExecutionProvider': {'tunable_op_enabled': '0', 'cudnn_conv_use_max_workspace': '1', 'enable_cuda_graph': '0', 'do_copy_in_default_stream': '1', 'arena_extend_strategy': 'kNextPowerOfTwo', 'cudnn_conv1d_pad_to_nc1d': '0', 'gpu_external_empty_cache': '0', 'gpu_external_free': '0', 'gpu_external_alloc': '0', 'gpu_mem_limit': '18446744073709551615', 'cudnn_conv_algo_search': 'EXHAUSTIVE', 'device_id': '0'}}
model ignore: /home/jianming_ge/.insightface/models/buffalo_l/1k3d68.onnx landmark_3d_68
Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}, 'CUDAExecutionProvider': {'tunable_op_enabled': '0', 'cudnn_conv_use_max_workspace': '1', 'enable_cuda_graph': '0', 'do_copy_in_default_stream': '1', 'arena_extend_strategy': 'kNextPowerOfTwo', 'cudnn_conv1d_pad_to_nc1d': '0', 'gpu_external_empty_cache': '0', 'gpu_external_free': '0', 'gpu_external_alloc': '0', 'gpu_mem_limit': '18446744073709551615', 'cudnn_conv_algo_search': 'EXHAUSTIVE', 'device_id': '0'}}
model ignore: /home/jianming_ge/.insightface/models/buffalo_l/2d106det.onnx landmark_2d_106
Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}, 'CUDAExecutionProvider': {'tunable_op_enabled': '0', 'cudnn_conv_use_max_workspace': '1', 'enable_cuda_graph': '0', 'do_copy_in_default_stream': '1', 'arena_extend_strategy': 'kNextPowerOfTwo', 'cudnn_conv1d_pad_to_nc1d': '0', 'gpu_external_empty_cache': '0', 'gpu_external_free': '0', 'gpu_external_alloc': '0', 'gpu_mem_limit': '18446744073709551615', 'cudnn_conv_algo_search': 'EXHAUSTIVE', 'device_id': '0'}}
find model: /home/jianming_ge/.insightface/models/buffalo_l/det_10g.onnx detection [1, 3, '?', '?'] 127.5 128.0
Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}, 'CUDAExecutionProvider': {'tunable_op_enabled': '0', 'cudnn_conv_use_max_workspace': '1', 'enable_cuda_graph': '0', 'do_copy_in_default_stream': '1', 'arena_extend_strategy': 'kNextPowerOfTwo', 'cudnn_conv1d_pad_to_nc1d': '0', 'gpu_external_empty_cache': '0', 'gpu_external_free': '0', 'gpu_external_alloc': '0', 'gpu_mem_limit': '18446744073709551615', 'cudnn_conv_algo_search': 'EXHAUSTIVE', 'device_id': '0'}}
model ignore: /home/jianming_ge/.insightface/models/buffalo_l/genderage.onnx genderage
Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}, 'CUDAExecutionProvider': {'tunable_op_enabled': '0', 'cudnn_conv_use_max_workspace': '1', 'enable_cuda_graph': '0', 'do_copy_in_default_stream': '1', 'arena_extend_strategy': 'kNextPowerOfTwo', 'cudnn_conv1d_pad_to_nc1d': '0', 'gpu_external_empty_cache': '0', 'gpu_external_free': '0', 'gpu_external_alloc': '0', 'gpu_mem_limit': '18446744073709551615', 'cudnn_conv_algo_search': 'EXHAUSTIVE', 'device_id': '0'}}
model ignore: /home/jianming_ge/.insightface/models/buffalo_l/w600k_r50.onnx recognition
set det-size: (640, 640)
faces:::: [{'bbox': array([105.29808,  83.55866, 218.93301, 230.41777], dtype=float32), 'kps': array([[137.7511 , 151.92409],
       [190.89203, 143.08504],
       [171.97752, 181.22136],
       [155.1662 , 205.49452],
       [192.61273, 197.9723 ]], dtype=float32), 'det_score': 0.8469027}]

人脸检测的效果：

文件目录树也贴上来供参考：
![在这里插入图片描述](https://img-blog.csdnimg.cn/591c2829601e4a4183c437125c8f569c.png

1.3 人脸特征抽取

1.3.1 模型下载

https://github.com/deepinsight/insightface/tree/master/model_zoo
一般说来，模型越大，速度越慢，准确度越高
这个页面上的任何一个权重都可以用，我这里下载后重命名为recg.onnx
这是git 的样例，但其实是不全的。
新建bbb.py

"""
Call Recognition Models
"""
import cv2
import numpy as np
import insightface
from insightface.app import FaceAnalysis
from insightface.data import get_image as ins_get_image

handler = insightface.model_zoo.get_model('your_recognition_model.onnx')
handler.prepare(ctx_id=0)

刚开始运行肯定报错，因为你没有your_recognition_model.onnx 的权重，
通过代码追踪：它应该放在 ~/.insightface/models/your_recognition_model.onnx

1.3.2 重新跑一下检测和识别

而且权重应该放在项目目录下：

bbb.py 更新为：

"""
Call Recognition Models
"""
import cv2
import numpy as np
import insightface
from insightface.app import FaceAnalysis
from insightface.data import get_image as ins_get_image

app = FaceAnalysis(allowed_modules=['detection'],providers=['CUDAExecutionProvider', 'CPUExecutionProvider'],download=False)
app.prepare(ctx_id=0, det_size=(640, 640))
img = ins_get_image('ldh')  #不用带后缀，图片放到./insightface/python-package/insightface/data/images
faces = app.get(img)
print("faces::::", faces)

handler = insightface.model_zoo.get_model('w600k_r50.onnx')
handler.prepare(ctx_id=0)
img = ins_get_image('ldh')
feature = handler.get(img,faces[0])
print("size 0f feature:", len(feature))
print("feature:",feature)

注意点：实际在使用过程中，

1. 先做人脸检测
2. 做特征抽取
3. 再做相似度计算

1.3.3 人脸监测返回值分析

人脸监测的返回值, 单张人脸：

faces:::: [{'bbox': array([398.00247,  76.93284, 517.12415, 217.21019], dtype=float32), 'kps': array([[420.43942, 127.73588],
       [472.40298, 119.96787],
       [440.8788 , 139.35797],
       [426.87814, 178.95143],
       [473.05652, 172.67671]], dtype=float32), 'det_score': 0.82732844}]

faces:::: [{'bbox': array([151.52905 ,   9.983715, 252.05127 , 149.92014 ], dtype=float32), 'kps': array([[179.53656 ,  65.759674],
       [225.35909 ,  66.749855],
       [202.66101 ,  89.19701 ],
       [176.11479 , 110.48839 ],
       [225.03027 , 111.73684 ]], dtype=float32), 'det_score': 0.903729}, {'bbox': array([390.146   ,  19.774494, 489.32114 , 154.33212 ], dtype=float32), 'kps': array([[417.48557 ,  73.57366 ],
       [463.98648 ,  72.462814],
       [442.4967  , 100.443985],
       [420.76508 , 118.47941 ],
       [463.62903 , 117.62505 ]], dtype=float32), 'det_score': 0.89457685}]
=============

返回一个list， len是人脸的数量
每个单元包含：
bbox 人脸框坐标
kps 人脸关键点的坐标 左眼、右眼、鼻子、嘴巴左侧、嘴巴右侧
det_score 检测的分数

人脸特征是返回一个512纬的向量：

size 0f feature: 512
feature: [-2.65807837e-01 -5.98709404e-01  1.44702032e-01  2.63355345e-01
  4.07146290e-02 -1.62199542e-01  2.32072305e-02 -2.41173565e-01
 -3.18579137e-01 -5.12303770e-01  4.71155867e-02  2.73445308e-01
 -8.21143925e-01  1.60933450e-01  2.68835127e-02 -1.19085282e-01
 -9.30740461e-02 -1.72320172e-01 -2.14248434e-01  2.86992230e-02
  3.17545533e-01  1.75784290e-01 -1.22256204e-01 -1.63961858e-01
 -8.66541564e-01  7.55451992e-02  2.67704695e-01 -5.87792039e-01
 -1.69785097e-02 -2.07043514e-02  5.06040215e-01  2.22781166e-01
 -2.63346657e-02  8.10962379e-01 -3.26032043e-01  1.26174808e-01
  3.17236811e-01  6.22351229e-01 -3.11159819e-01  4.34068412e-01
 -2.57273525e-01  2.18948558e-01  5.02837479e-01 -3.66360933e-01
 -4.49983001e-01  3.29971194e-01 -4.58328962e-01 -2.24694684e-01
 -6.20792098e-02 -2.27833763e-01 -2.73626387e-01 -1.57144777e-02
  7.91971147e-01  1.46265015e-01 -5.00584424e-01 -3.09522271e-01
  6.18827194e-02 -3.79446059e-01  4.31523114e-01 -4.84285116e-01
 -3.52546960e-01 -7.49332607e-02  6.07803948e-02  8.62523675e-01
 -4.10395920e-01 -2.44194195e-01 -3.41306150e-01 -8.33527073e-02
  5.68045788e-02 -1.97309047e-01  1.42464489e-01 -6.44236326e-01
  3.12686771e-01 -2.39409551e-01  4.39812720e-01 -4.36708421e-01
 -4.98010159e-01  1.31110698e-01 -5.03139734e-01 -4.52581614e-01
 -8.60964954e-02  1.03808708e-01 -3.57752778e-02  8.27392399e-01
  3.43307912e-01 -5.30499697e-01  2.62764663e-01  4.72977400e-01
  2.66696811e-01 -2.69342978e-02 -5.43406844e-01  3.88814628e-01
  8.32139477e-02 -7.24730372e-01  4.20132369e-01 -4.74645436e-01
 -1.48665071e-01  9.54597652e-01 -4.56728972e-02  3.51921208e-02
  3.72070462e-01 -1.53715964e-02 -8.53569135e-02 -8.37187350e-01
 -4.30278433e-03 -9.07578766e-02 -2.80029148e-01 -3.56460154e-01
 -6.20279536e-02  4.94443417e-01  1.15014195e+00 -4.20974493e-01
  4.80442680e-02 -4.37882274e-01 -1.71415120e-01 -3.32768500e-01
 -1.02889575e-01  2.16156662e-01  7.01144218e-01 -4.51926380e-01
 -4.58793879e-01  1.81827918e-02 -1.88612550e-01 -1.14988470e-02
 -1.77762285e-01 -2.05713645e-01  6.62348121e-02  1.67144701e-01
 -6.38863564e-01 -1.04559563e-01  2.81668335e-01  2.02162907e-01
 -7.78863609e-01  3.30200553e-01  5.50253451e-01  9.45175365e-02
  2.95698971e-01 -3.63492787e-01  3.95007521e-01  4.72791910e-01
 -1.13705151e-01  3.27019036e-01 -2.52318531e-01  5.03111146e-02
  7.85713866e-02 -4.40496325e-01 -1.02330513e-01 -2.04051405e-01
 -4.25574929e-01  1.15317412e-01 -2.48718798e-01  1.49252310e-01
  1.05377071e-01 -1.19814776e-01  5.92564166e-01  3.30846272e-02
  7.21025541e-02 -3.52899522e-01  7.61120737e-01 -6.52381837e-01
  6.50658727e-01 -5.15108407e-01 -1.94832787e-01 -4.66580153e-01
 -3.02693993e-01  3.60020995e-01 -6.04675889e-01  3.48266721e-01
  1.35741889e-01 -2.12851986e-01 -7.17316985e-01  1.31764725e-01
  2.28615105e-01  5.24750538e-02 -4.72336970e-02 -6.26476049e-01
 -4.56452340e-01  1.02681197e-01  3.58329445e-01 -3.19489807e-01
 -3.44365656e-01 -4.40581203e-01 -1.41268568e-02  6.94410443e-01
 -2.67537236e-01 -7.72025576e-03 -5.61997116e-01 -7.19041407e-01
  1.20449759e-01  6.45162582e-01  3.46629024e-01  1.10950135e-01
  8.20405841e-01  2.94094980e-01 -1.51019976e-01 -5.04539907e-02
 -7.21346065e-02  2.75275677e-01  4.68920708e-01 -5.11016130e-01
  1.87380016e-01  8.27821791e-02 -3.10316354e-01 -1.03935346e-01
  5.48591197e-01  9.24001709e-02 -3.89272362e-01 -1.15961254e+00
 -4.71911073e-01  3.22682679e-01 -3.27484250e-01 -2.13711888e-01
  8.46880302e-02  1.13394640e-01 -2.57271856e-01  4.57801521e-01
 -1.56833291e-01 -4.63344246e-01 -5.22429049e-02  1.63254842e-01
  3.00276607e-01  2.76694864e-01 -3.10461104e-01 -5.87711275e-01
 -1.89284272e-02  4.10056353e-01  6.57250285e-01 -8.84221718e-02
 -3.13798636e-01  5.00106215e-01 -4.48331796e-02 -5.13528764e-01
 -9.17405933e-02  7.00492561e-01 -2.52461191e-02 -4.65177923e-01
  3.35291862e-01  1.67592824e-01 -2.38635331e-01  6.97482347e-01
  1.63971394e-01 -3.29721011e-02 -1.26637876e-01  4.27806467e-01
 -5.34400046e-01 -2.61363145e-02  3.35392177e-01 -4.52100456e-01
 -1.15982279e-01 -4.80096698e-01 -1.20243188e-02 -8.81862491e-02
 -1.41368508e-01  8.21686313e-02  2.38767043e-01 -8.79521295e-02
  1.31764039e-01  9.93780792e-02  3.97302061e-01  2.62562603e-01
  5.31454384e-01  1.18952766e-01  1.63879409e-01  2.79800206e-01
  1.71809718e-01 -9.73480046e-02  5.50766960e-02  2.19342694e-01
  3.70924145e-01  2.22219989e-01 -1.48917332e-01  2.90967226e-01
 -1.31165534e-01 -1.25355244e-01  1.72034904e-01 -1.20657332e-01
  1.64537430e-01 -2.72813380e-01  4.12511617e-01  3.81849200e-01
 -3.10269538e-02 -5.84295869e-01 -1.62318975e-01  4.32897478e-01
 -2.25531310e-02  1.45282999e-01 -6.18601479e-02 -2.70079710e-02
  1.48326024e-01 -6.03496656e-02 -6.77019823e-03 -5.22333384e-01
 -1.42824680e-01  1.05511181e-01 -1.57965735e-01 -1.94174737e-01
  5.04310012e-01 -6.04734868e-02  3.38690877e-01  2.85632819e-01
  2.44570196e-01  4.41604286e-01 -2.50822127e-01 -3.39492172e-01
 -4.40223627e-02 -7.41581142e-01 -1.18939653e-01  1.32609561e-01
  5.79564087e-02 -1.12131372e-01 -3.30193996e-01  1.77734077e-01
 -8.44704509e-02  8.52265060e-02  7.22285330e-01 -7.39744082e-02
 -3.24919313e-01  1.52587101e-01  2.60848194e-01  4.75044131e-01
  8.96252871e-01 -7.45972931e-01 -1.91705048e-01 -5.63617408e-01
  4.07978743e-01 -5.42383134e-01  5.90304434e-01 -1.02651024e+00
  1.07888348e-01 -3.20087671e-02 -6.86910003e-02  4.59433228e-01
 -1.34848386e-01  1.79632828e-01 -6.78926706e-01  1.82937503e-01
 -6.39753282e-01 -2.47191161e-01 -1.42870158e-01 -3.90086859e-01
  2.62941182e-01 -5.99119365e-02 -9.82555468e-03 -2.31434792e-01
  4.23140265e-02 -2.33853012e-01  7.04509556e-01 -4.41635907e-01
 -2.51954973e-01 -3.28132778e-01  7.64646053e-01 -3.54267150e-01
  5.39237082e-01  1.79837737e-02  1.73653990e-01  1.45106345e-01
 -2.72235245e-01 -5.94357669e-01 -3.84668678e-01 -2.96734065e-01
  8.66554603e-02  5.53680919e-02 -7.88553357e-01  1.36896238e-01
 -4.14724737e-01 -1.44677877e-01  1.79900452e-01 -1.39882132e-01
  2.18250602e-01  6.80885389e-02 -7.68631250e-02 -7.89565086e-01
  3.48717213e-01  5.64985991e-01  5.58483787e-02 -8.17401558e-02
  2.03474388e-01 -1.07692860e-01  6.84533358e-01  3.19718868e-01
  1.21635854e-01  3.41574609e-01 -1.78951304e-02  2.78006196e-01
  1.80511326e-02 -6.24188110e-02 -6.32650331e-02  1.31553754e-01
  1.79080918e-01  1.01404369e-01  4.41990018e-01  9.36895087e-02
  7.87015259e-02  2.14785039e-01 -5.45855701e-01  2.24413946e-01
  4.32758123e-01 -4.15375143e-01  2.15920016e-01  2.70790398e-01
  6.58806413e-02  1.41609073e-01  3.54366191e-02 -3.72616023e-01
  3.92095059e-01 -9.02478099e-02  4.66177404e-01 -6.86050534e-01
 -3.16491425e-01  4.46931541e-01 -2.10206121e-01  6.28514409e-01
 -2.65414834e-01 -3.76454681e-01 -3.56145829e-01 -1.18043542e-01
  5.63065000e-02  6.98566914e-01 -3.88334543e-01  5.95293581e-01
  6.92259550e-01  2.15426758e-01  6.51695311e-01  1.98373515e-02
 -1.47576705e-01 -3.12872767e-01  4.05580014e-01  2.66416162e-01
  1.96047127e-01 -1.27053857e-01  1.09347150e-01  3.83434832e-01
  1.69589058e-01 -3.51539433e-01  2.03101888e-01  5.83723664e-01
 -5.45567989e-01  8.64841640e-02  1.94809675e-01 -2.73879915e-01
  5.09571843e-02  9.19594914e-02 -2.97315329e-01 -2.41457775e-01
  6.12340942e-02  1.42051131e-01  3.24435651e-01 -7.38600254e-01
 -4.35348183e-01  1.50280312e-01  2.81985532e-02  5.68533838e-01
  2.32664168e-01  3.33400816e-01  1.46391779e-01 -5.85084502e-03
 -1.55448794e-01 -4.67291296e-01 -4.34049487e-01 -3.40150416e-01
  1.04534440e-01 -4.27028328e-01  2.94970989e-01  3.17830712e-01
  5.78037322e-01 -2.43507465e-03  6.16211236e-01 -1.76521957e-01
 -1.05725817e-01 -2.66839921e-01  6.76848069e-02  2.33130768e-01
  3.24840128e-01  1.93292648e-01 -1.03790827e-01 -3.10859770e-01
  1.08004078e-01  4.30708565e-02 -1.50686622e-01 -3.32239121e-01
 -7.74850786e-01  8.22230577e-02 -2.64589787e-01 -1.18717141e-01
 -1.79947644e-01  2.60864738e-02  7.45431662e-01 -6.43727034e-02
 -1.80508792e-01 -5.09034805e-02 -1.89890876e-01  1.10911153e-01
  3.64015251e-01 -2.90384492e-07  4.50303376e-01  1.55005336e-01
 -4.08384293e-01  4.87070352e-01  1.31157666e-01 -1.15866172e+00
 -2.70245939e-01 -4.77115929e-01  8.44599679e-02  3.13389778e-01
 -3.81731451e-01  3.79520021e-02  4.61172126e-02  5.15780486e-02
  2.22334802e-01 -6.06625915e-01 -2.58048952e-01 -1.14576712e-01]

1.3.4 计算相似度

相似度计算就是用余弦函数进行计算，计算出相似度，其实就是识别的内容。

 # 计算相似度
 similarity = np.dot(feature1, feature2.T)

1.4 全流程的相似度

mmp 搞来搞去，这里有一个pipline

import argparse
import cv2
import sys
import numpy as np
import insightface
from insightface.app import FaceAnalysis
from insightface.data import get_image as ins_get_image

assert insightface.__version__>='0.3'

parser = argparse.ArgumentParser(description='insightface app test')
# general
parser.add_argument('--ctx', default=0, type=int, help='ctx id, <0 means using cpu')
parser.add_argument('--det-size', default=640, type=int, help='detection size')
args = parser.parse_args()

app = FaceAnalysis()
app.prepare(ctx_id=args.ctx, det_size=(args.det_size,args.det_size))

img = ins_get_image('t1')
faces = app.get(img)
assert len(faces)==6
rimg = app.draw_on(img, faces)
cv2.imwrite("./t1_output.jpg", rimg)

# then print all-to-all face similarity
feats = []
for face in faces:
    feats.append(face.normed_embedding)
feats = np.array(feats, dtype=np.float32)
sims = np.dot(feats, feats.T)
print(sims)

执行命令：python ccc.py --ctx 0 输出，分数是1的代表最相似

[[ 1.0000002   0.06442557 -0.01041015 -0.08265568  0.03555955 -0.02871545]
 [ 0.06442557  1.0000002   0.02620501  0.09680527 -0.02184767  0.21340981]
 [-0.01041015  0.02620501  0.99999994  0.14635125  0.06818002 -0.03542848]
 [-0.08265568  0.09680527  0.14635125  1.0000001  -0.0301404  -0.01418785]
 [ 0.03555955 -0.02184767  0.06818002 -0.0301404   0.99999994  0.05765504]
 [-0.02871545  0.21340981 -0.03542848 -0.01418785  0.05765504  0.99999964]]

6个人都比一遍，肯定对角线最相似啊，内部是不是已经做了人脸矫正和对齐，就不得而知了，懂的大神可以留言一下。
跑到这里，其实就差业务化了

2. 业务化人脸识别

敬请期待
-----分割线以上是20230529完成
20230531----md 昨天干了一天杂活儿，终于有时间搞这个了。

2.1 思路：

1.人脸库图片导入，提前抽取后人脸特征向量，并存储起来，可以使.txt, 或者 redis，或者 mysql
人脸库要求，一张图片一张脸
2. 项目启动时，把人脸库加载进内存，等待要识别的人脸图片送入
3. 获取送入的图片，通过人脸监测模型，获取人脸的数量，并抽取每个人脸（送入图片可能包含N张人脸）的特征
4. 以循环的方式和数据库人脸作比对，送入图片的每张，和库里的每个人脸计算相似度，并且排名。
5. 获取每个人脸的相似度排行，满足阈值的，返回识别成功，并标注到原图上。

2.2 开干

项目路径

face_db 是人脸库，这里只有郭、刘两位天王

2.2.1 遍历人脸库，获取人脸库的特征向量

只在项目启动时做一次，或者保存在数据库中

def get_face_db_features(face_db_dir=""):
    """
    获取人脸db的features
    """
    face_db_dir = os.path.join(os.path.dirname(os.path.abspath(__file__)),"face_db")
    img_gen = glob.glob(face_db_dir+"/*")
    args = argparse.Namespace()
    assert len(img_gen)!=0,"人脸库图片数量为0"
    face_db_list = []
    for img_path in img_gen:
         args.img1 = img_path
         image1 = cv2.imread(args.img1)
         bboxes1, kpss1 = detector.autodetect(image1, max_num=1)
         assert bboxes1.shape[0]!=0, f"图片:{img_path}未检测到人脸"
         assert len(bboxes1) == 1,f"图片:{img_path}人脸数量大于1"
         kps1 = kpss1[0]
         feat1 = rec.get(image1, kps1)
         # 文件名即为人脸名字
         name = os.path.splitext(os.path.basename(img_path))[0]
         face_db_list.append(dict(name = name,feat= feat1))
    return face_db_list

对人脸库的要求苛刻，需要每张图片只有一张人脸，图片名称即为人员名称。
这个face_db_list 就是郭刘两天王的人脸特征。

2.2.2 模拟输入一张图片，计算距离

因为输入的图片可能是无人脸，1张，或者多张人脸。
这里做难度最大的多张人脸。
也就是说，这里面有三种可能：1.没有两个天王，2.有其中之一，3.两个都有
少废话，先看效果：

def get_simility_face(img_path:str = ""):
    args = argparse.Namespace()
    args.img1 = img_path
    image1 = cv2.imread(args.img1)
    # 这里先写死最多容纳10个人脸
    bboxes2, kpss2 = detector.autodetect(image1, max_num=10)
    # 等待识别的人脸
    recognition_face_list = []
    for idx,kps2 in enumerate(kpss2):
        feat2 = rec.get(image1, kps2)
        recognition_face_list.append(dict(bbox = bboxes2[idx], ksp = kps2, feat = feat2))
    # 代码运行到这里, face_db 和 recognition_db 都准备好了，现在需要计算距离
    result = []
    for idx, wait_face in enumerate(recognition_face_list):
        temp_list = []
        for face_db in  face_db_list:
            name = face_db["name"]
            feat1 = face_db["feat"]
            feat2 = wait_face["feat"]
            sim = rec.compute_sim(feat1, feat2)
            if sim>=0.28:
                temp_list.append(dict(idx=idx,name=name,sim=sim))
        if temp_list:
            result.append(temp_list)    
    # 到这里,获取了一result 其数据结构为：
    """
    送入了一张包含人脸很多的照片,包含了10张人脸
    人脸库2张，ldh和gfc
    [
        [{'idx': 0, 'name': 'gfc', 'sim': 0.07851476}, {'idx': 0, 'name': 'ldh', 'sim': -0.06523238}], 
        [{'idx': 1, 'name': 'gfc', 'sim': -0.05163587}, {'idx': 1, 'name': 'ldh', 'sim': -0.032635976}], 
        [{'idx': 2, 'name': 'gfc', 'sim': -0.012458734}, {'idx': 2, 'name': 'ldh', 'sim': -0.049863357}], 
        [{'idx': 3, 'name': 'gfc', 'sim': 0.18950702}, {'idx': 3, 'name': 'ldh', 'sim': -0.03275803}], 
        [{'idx': 4, 'name': 'gfc', 'sim': 0.0027918643}, {'idx': 4, 'name': 'ldh', 'sim': -0.06449092}], 
        [{'idx': 5, 'name': 'gfc', 'sim': 0.17510812}, {'idx': 5, 'name': 'ldh', 'sim': 0.5763208}], 
        [{'idx': 6, 'name': 'gfc', 'sim': 0.008685053}, {'idx': 6, 'name': 'ldh', 'sim': 0.029168786}], 
        [{'idx': 7, 'name': 'gfc', 'sim': -0.06675171}, {'idx': 7, 'name': 'ldh', 'sim': 0.026193036}], 
        [{'idx': 8, 'name': 'gfc', 'sim': -0.1017633}, {'idx': 8, 'name': 'ldh', 'sim': 0.048399623}], 
        [{'idx': 9, 'name': 'gfc', 'sim': -0.009450468}, {'idx': 9, 'name': 'ldh', 'sim': -0.029955076}]
    ]
    """
    if not result:
        return []
    # 按 sim 从大到小排序，每一行，只选取最大的一个
    sorted_data = sorted(result, key=lambda x: x[0]['sim'], reverse=True)
    dimg = image1.copy()
    font = cv2.FONT_HERSHEY_SIMPLEX
    for row in sorted_data:
        face_info = row[0]
        idx = face_info["idx"]
        bbox = recognition_face_list[idx]["bbox"]
        box = bbox.astype(np.int32)
        color = (0, 0, 255)
        cv2.rectangle(dimg, (box[0], box[1]), (box[2], box[3]), color, 2)
        cv2.putText(dimg, f"ID: {idx}", (box[0] + 10, box[1] + 30), font, 0.5, (0, 0, 255), 2)
    cv2.imwrite("./t1_output.jpg", dimg)

核心逻辑就是挨个对比，>0.28的算是同一个人，但是为以防万一，我会把>0.28的人里面选一个分数最高的。
比如，input image 有10张脸， 其中一张 和人脸库的好几张脸都相似， 那么我会选一个分数最高的。这里的逻辑处理还是很复杂的。
后面根据返回结果给业务端，或者封装成web api的方式，就看诸位自己了。大家加油

3. ffmpeg 进行拉流抽帧

opencv也可以做，但是opencv比ffmpeg耗费资源，因此这次要用ffmpeg来做
参考ref：https://blog.csdn.net/asialee_bird/article/details/129014872

3.1 安装ffmpeg

要在CentOS上安装FFmpeg，您需要执行以下步骤：

1. 打开终端并以root用户身份登录。

2. 安装EPEL repository：

yum install epel-release

3. 安装Nux Dextop repository：

rpm -Uvh http://li.nux.ro/download/nux/dextop/el7/x86_64/nux-dextop-release-0-1.el7.nux.noarch.rpm

4. 安装FFmpeg及相关库：

yum install ffmpeg ffmpeg-devel

5. 检查FFmpeg是否正确安装：

ffmpeg -version

如果出现FFmpeg版本号，则说明成功安装。

6. （可选）安装其他用于特定目的的库，例如libvpx、libmp3lame等。

3.2 视频流抽帧

rtsp流抽帧。参数说明：-y ：如果提示文件已经存在，自动执行y，对已存在文件进行覆盖；-f image2：以图片格式保存；-r 1：帧数，每秒截取一帧，-r 1/5 表示每5秒抽取一帧；img%03d.jpg 格式化命名，会生成img001.jpg,img002.jpg

ffmpeg -i "rtsp://dxcs:cs123456@200.36.43.131:554/Streaming/Channels/1001" -y -f image2 -r 1/5 img%03d.jpg

rtsp流抽帧，tcp协议推流需要加上: -rtsp_transport tcp

ffmpeg -rtsp_transport tcp -i "rtsp://dxcs:cs123456@200.36.43.131:554/Streaming/Channels/1001" -y -f image2 -r 1/5 img%03d.jpg

两种任选其一即可，这样我们可以用python来操作ffmpeg，并且让它返回图片路径，送入到模型，进行人脸识别！

import subprocess
import os

import subprocess
import os
import time

def extract_frames(video_path, output_dir, frame_rate="1/5"):
    """
    使用FFmpeg从视频中提取帧。
    :param video_path: 视频文件路径
    :param output_dir: 输出帧的目录
    :param frame_rate: 提取帧的间隔，默认为每秒提取2帧
    :return: 提取的帧图像文件路径
    """
    # 构造FFmpeg命令
    cmd = ["ffmpeg", "-i", video_path, "-vf", "fps={}".format(frame_rate), "{}/%010d.jpg".format(output_dir)]
    # cmd = ["ffmpeg", "-i", video_path, "-vf", "fps={}".format(frame_rate), "{}/{}.jpg".format(output_dir,int(time.time()))]
    # 执行命令
    subprocess.call(cmd)



if __name__ == "__main__":
    video_path = "rtsp://admin:123456@172.18.45.15:554/h264/ch1/main/av_stream"
    output_dir = "./video_img"
    frame_rate = "1/5"
    extract_frames(video_path, output_dir, frame_rate)

另一个文件监听video_img的文件变化，并写入到img_log日志

import time
import os
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler

class MyHandler(FileSystemEventHandler):
    def on_created(self, event):
        if event.is_directory:  # 忽略文件夹的创建事件
            return
        if not event.is_directory:
            filepath = event.src_path
            # 在这里可以写入文件,另外的线程定于这个文件即可
            print(f"New file detected: {filepath}")
            with open(img_log,mode="a") as f:
                f.write(filepath + os.linesep)

if __name__ == "__main__":
    base_dir =  os.path.dirname(os.path.abspath(__file__))
    img_log = os.path.join(base_dir,"img.log")
    folder_to_watch = os.path.join(base_dir,"video_img")  # 监听文件夹的路径

    event_handler = MyHandler()
    observer = Observer()
    observer.schedule(event_handler, folder_to_watch, recursive=False)
    observer.start()

    try:
        while True:
            time.sleep(1)
    except KeyboardInterrupt:
        observer.stop()
    observer.join()

4.进阶

把人脸更多的结构化属性带进来：
年龄、性别、表情、眼睛开闭、嘴巴开闭、是否带眼镜、是否带口罩、是否有胡子…
这些我还搞不清楚是不是一个模型，但应该不是一个。旷世有的模型能识别年龄和性别，会的大神欢迎留言给点指导啊！
应该是先把做人脸检测，把人脸框出来，再送入第二个模型，做分类，是否有胡子，是否戴口罩。。。
嘴巴/眼睛的开闭用关键点计算即可。

总结

我觉得人脸矫正、人脸对齐，这个insight的库都帮我们做了。真是太好用了。