音频驱动人脸2d显示

目录

video-retalking

驱动后渲染成2d人脸

提供下载模型

---

video-retalking

GitHub - OpenTalker/video-retalking: [SIGGRAPH Asia 2022] VideoReTalking: Audio-based Lip Synchronization for Talking Head Video Editing In the Wild

驱动后渲染成2d人脸

GitHub - yerfor/Real3DPortrait: Real3D-Portrait: One-shot Realistic 3D Talking Portrait Synthesis; ICLR 2024 Spotlight; Official code

提供下载模型

deep_3drecon/BFM/
├── 01_MorphableModel.mat
├── BFM_exp_idx.mat
├── BFM_front_idx.mat
├── BFM_model_front.mat
├── Exp_Pca.bin
├── facemodel_info.mat
├── index_mp468_from_mesh35709.npy
├── mediapipe_in_bfm53201.npy
└── std_exp.txt
Pre-trained Real3D-Portrait
Download Pre-trained Real3D-Portrait：Google Drive or BaiduYun Disk with Password 6x4f

Put the zip files in checkpoints and unzip them, the file structure will be like this:

checkpoints/
├── 240210_real3dportrait_orig
│   ├── audio2secc_vae
│   │   ├── config.yaml
│   │   └── model_ckpt_steps_400000.ckpt
│   └── secc2plane_torso_orig
│       ├── config.yaml
│       └── model_ckpt_steps_100000.ckpt
└── pretrained_ckpts
    └── mit_b0.pth

GitHub - yerfor/GeneFacePlusPlus: GeneFace++: Generalized and Stable Real-Time 3D Talking Face Generation; Official Code