C# 3D人脸重建，人头姿势估计

介绍

github地址：

https://github.com/cleardusk/3DDFA_V2

Introduction

This work extends 3DDFA, named 3DDFA_V2, titled Towards Fast, Accurate and Stable 3D Dense Face Alignment, accepted by ECCV 2020. The supplementary material is here. The gif above shows a webcam demo of the tracking result, in the scenario of my lab. This repo is the official implementation of 3DDFA_V2.

Compared to 3DDFA, 3DDFA_V2 achieves better performance and stability. Besides, 3DDFA_V2 incorporates the fast face detector FaceBoxes instead of Dlib. A simple 3D render written by c++ and cython is also included. This repo supports the onnxruntime, and the latency of regressing 3DMM parameters using the default backbone is about 1.35ms/image on CPU with a single image as input. If you are interested in this repo, just try it on this google colab! Welcome for valuable issues, PRs and discussions 😄

效果

模型信息

Inputs
-------------------------
name：input
tensor：Float[-1, 3, 120, 120]
---------------------------------------------------------------

Outputs
-------------------------
name：camera_matrix
tensor：Float[-1, 3, 4]
name：landmarks
tensor：Float[-1, 68, 2]
---------------------------------------------------------------

项目

VS2022

.net framework 4.8

OpenCvSharp 4.8

Microsoft.ML.OnnxRuntime 1.16.2

代码

using OpenCvSharp;
using System;
using System.Collections.Generic;
using System.Drawing;
using System.Windows.Forms;
 
namespace Onnx_Demo
{
    public partial class frmMain : Form
    {
        public frmMain()
        {
            InitializeComponent();
        }
 
        string fileFilter = "*.*|*.bmp;*.jpg;*.jpeg;*.tiff;*.tiff;*.png";
        string image_path = "";
 
        DateTime dt1 = DateTime.Now;
        DateTime dt2 = DateTime.Now;
 
        Mat image;
 
        private void button1_Click(object sender, EventArgs e)
        {
            OpenFileDialog ofd = new OpenFileDialog();
            ofd.Filter = fileFilter;
            if (ofd.ShowDialog() != DialogResult.OK) return;
 
            pictureBox1.Image = null;
            pictureBox2.Image = null;
            textBox1.Text = "";
 
            image_path = ofd.FileName;
            pictureBox1.Image = new Bitmap(image_path);
            image = new Mat(image_path);
        }
 
        private void Form1_Load(object sender, EventArgs e)
        {
         
        }
 
        private void button2_Click(object sender, EventArgs e)
        {
            if (image_path == "")
            {
                return;
            }
            textBox1.Text = "检测中，请稍等……";
            pictureBox2.Image = null;
            Application.DoEvents();
 
            image = new Mat(image_path);
 
            Detect_Face detect_net = new Detect_Face(0.7f);
            Face_Mesh mesh_net = new Face_Mesh("pose");//choices=["pose", "sparse"]
 
            dt1 = DateTime.Now;
            List<BoxInfo> bboxes = detect_net.detect(image);
 
            foreach (var item in bboxes)
            {
                mesh_net.detect(image, item);
            }
            dt2 = DateTime.Now;
 
            pictureBox2.Image = new Bitmap(image.ToMemoryStream());
            textBox1.Text = "推理耗时:" + (dt2 - dt1).TotalMilliseconds + "ms";
 
        }
 
        private void pictureBox2_DoubleClick(object sender, EventArgs e)
        {
            Common.ShowNormalImg(pictureBox2.Image);
        }
 
        private void pictureBox1_DoubleClick(object sender, EventArgs e)
        {
            Common.ShowNormalImg(pictureBox1.Image);
        }
    }
}