【Paddle笔记】体验PaddleBoBo虚拟主播自动播报快速生成项目

@[TOC] 【Paddle笔记】体验PaddleBoBo虚拟主播自动播报快速生成项目

1、下载PaddleBoBo

1.1 HTML下载

Github 软件仓下载 PaddleBoBo

1.2 Git 下载

git clone https://github.com/JiehangXie/PaddleBoBo.git

(base) xf@VP01:~/ai$ git clone https://github.com/JiehangXie/PaddleBoBo.git
Cloning into 'PaddleBoBo'...
fatal: unable to access 'https://github.com/JiehangXie/PaddleBoBo.git/': GnuTLS recv error (-110): The TLS connection was non-properly terminated.

网络不畅，多试几次

(base) xf@VP01:~/ai$ git clone https://github.com/JiehangXie/PaddleBoBo.git
Cloning into 'PaddleBoBo'...
remote: Enumerating objects: 77, done.
remote: Counting objects: 100% (27/27), done.
remote: Compressing objects: 100% (15/15), done.
remote: Total 77 (delta 11), reused 23 (delta 10), pack-reused 50
Receiving objects: 100% (77/77), 9.58 MiB | 107.00 KiB/s, done.
Resolving deltas: 100% (19/19), done.
(base) xf@VP01:~/ai$ cd paddlebobo
(base) xf@VP01:~/ai/paddlebobo$ ll
total 52
drwxrwxrwx 1 xf xf   512 May  1 00:08 ./
drwxrwxrwx 1 xf xf   512 May  1 00:07 ../
drwxrwxrwx 1 xf xf   512 May  1 00:08 .git/
-rwxrwxrwx 1 xf xf 35149 May  1 00:08 LICENSE*
drwxrwxrwx 1 xf xf   512 May  1 00:08 PaddleTools/
-rwxrwxrwx 1 xf xf  3243 May  1 00:08 README.md*
-rwxrwxrwx 1 xf xf   463 May  1 00:08 config.py*
-rwxrwxrwx 1 xf xf   500 May  1 00:08 create_virtual_human.py*
-rwxrwxrwx 1 xf xf   286 May  1 00:08 default.yaml*
drwxrwxrwx 1 xf xf   512 May  1 00:08 docs/
drwxrwxrwx 1 xf xf   512 May  1 00:08 file/
-rwxrwxrwx 1 xf xf   875 May  1 00:08 general_demo.py*
-rwxrwxrwx 1 xf xf  2166 May  1 00:08 news_app.py*
-rwxrwxrwx 1 xf xf  1786 May  1 00:08 play.py*
-rwxrwxrwx 1 xf xf    35 May  1 00:08 requirements.txt*
(base) xf@VP01:~/ai/paddlebobo$

2、安装运行环境

2.1 Anaconda 环境管理

2.1.1 系统基础信息

Ubuntu 下基础环境搭建及CUDA配置参考 这里

• 查看操作系统
  uname -m && cat /etc/*release

(base) xf@VP01:~/ai/paddlebobo$ uname -m && cat /etc/*release
x86_64
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=22.04
DISTRIB_CODENAME=jammy
DISTRIB_DESCRIPTION="Ubuntu 22.04.2 LTS"
PRETTY_NAME="Ubuntu 22.04.2 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.2 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

• 查看系统CUDA版本
  nvidia-smi

+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 530.41.03              Driver Version: 531.41       CUDA Version: 12.1     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                  Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf            Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce RTX 4090         On | 00000000:01:00.0  On |                  Off |
|  0%   49C    P8               26W / 450W|   1150MiB / 24564MiB |      1%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

2.1.2 Python虚拟环境

• 为BoBo项目创建虚拟环境
  conda create -n ppai python=3.7
• 激活ppai虚拟环境
  conda activate ppai

(base) xf@VP01:~/ai/paddlebobo$ conda activate ppai
(ppai) xf@VP01:~/ai/paddlebobo$

2.2 安装Paddle框架

2.2.1 Paddle核心框架

飞桨官网点这里

• 安装与你硬件匹配的版本
  python -m pip install paddlepaddle-gpu==2.4.2.post117 -f https://www.paddlepaddle.org.cn/whl/linux/mkl/avx/stable.html

• 验证是否成功
  `python -c “import paddle; print(paddle.version);paddle.utils.run_check()”``

(ppai) xf@VP01:~/ai/paddlebobo$ python -c "import paddle; print(paddle.__version__);paddle.utils.run_check()"
2.4.2
Running verify PaddlePaddle program ...
W0501 02:12:52.942994 25793 gpu_resources.cc:61] Please NOTE: device: 0, GPU Compute Capability: 8.9, Driver API Version: 12.1, Runtime API Version: 11.7
W0501 02:12:52.965313 25793 gpu_resources.cc:91] device: 0, cuDNN Version: 8.8.
PaddlePaddle works well on 1 GPU.
PaddlePaddle works well on 1 GPUs.
PaddlePaddle is installed successfully! Let's start deep learning with PaddlePaddle now.

2.2.2 PaddleGAN && PaddleSpeech 依赖包

PaddleGAN 提供表情迁移和自动口型
PaddleSpeech 提供文字生成语音

pip install ppgan paddlespeech

3、虚拟主播语音播报

3.1 虚拟人参数配置

(ppai) xf@VP01:~/ai/paddlebobo$ cat default.yaml
GANDRIVING:
  FOM_INPUT_IMAGE: './file/input/test.png'
  FOM_DRIVING_VIDEO: './file/input/zimeng.mp4'
  FOM_OUTPUT_VIDEO: './file/input/test.mp4'

TTS:
  SPEED: 1.0
  PITCH: 1.0
  ENERGY: 1.0

SAVEPATH:
  VIDEO_SAVE_PATH: './file/output/video/'
  AUDIO_SAVE_PATH: './file/output/audio/'

• 虚拟人表情迁移配置
  GANDRIVING:
• 虚拟人原始图片
  FOM_INPUT_IMAGE: ‘./file/input/test.png’
• 真实主持人视频源（表情源）
  FOM_DRIVING_VIDEO: ‘./file/input/zimeng.mp4’
• 合成输出有表情的虚拟人视频的路径和文件名
  FOM_OUTPUT_VIDEO: ‘./file/input/test.mp4’

• 文字生成语音配置
  TTS:
• 语速
  SPEED: 1.0
• 音调
  PITCH: 1.0
• 发音能级
  ENERGY: 1.0

• 最终成品文件保存位置
  SAVEPATH:
  VIDEO_SAVE_PATH: ‘./file/output/video/’
  AUDIO_SAVE_PATH: ‘./file/output/audio/’

我的最终配置

GANDRIVING:
  FOM_INPUT_IMAGE: './file/input/my_mh.png'
  FOM_DRIVING_VIDEO: './file/input/zimeng.mp4