CosyVoice算法驱动的多语言数字人语音合成技术

克隆并安装
⦁    克隆仓库
git clone --recursive https://github.com/FunAudioLLM/CosyVoice.git

#进入下载好的文件夹
cd CosyVoice
git submodule update --init –recursive

安装 Conda：请参阅https://docs.conda.io/en/latest/miniconda.html
创建 Conda 环境：
conda create -n cosyvoice python=3.8

conda activate cosyvoice
# pynini is required by WeTextProcessing, use conda to install it as it can be executed on all platform.
conda install -y -c conda-forge pynini==2.1.5
pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host=mirrors.aliyun.com

# If you encounter sox compatibility issues
# ubuntu
sudo apt-get install sox libsox-dev
# centos
sudo yum install sox sox-devel

模型下载
强烈建议下载预训练CosyVoice-300M CosyVoice-300M-SFT CosyVoice-300M-Instruct模型和CosyVoice-ttsfrd资源。

# git模型下载，请确保已安装git lfs
mkdir -p pretrained_models
git clone https://www.modelscope.cn/iic/CosyVoice-300M.git pretrained_models/CosyVoice-300M
git clone https://www.modelscope.cn/iic/CosyVoice-300M-SFT.git pretrained_models/CosyVoice-300M-SFT
git clone https://www.modelscope.cn/iic/CosyVoice-300M-Instruct.git pretrained_models/CosyVoice-300M-Instruct
git clone https://www.modelscope.cn/iic/CosyVoice-ttsfrd.git pretrained_models/CosyVoice-ttsfrd

(或者，可以解压缩ttsfrd资源并安装ttsfrd包以获得更好的文本规范化性能。
注意，这一步不是必须的，如果没有安装ttsfrd，默认使用WeTextProcessing。)

cd pretrained_models/CosyVoice-ttsfrd/
unzip resource.zip -d .
pip install ttsfrd-0.3.6-cp38-cp38-linux_x86_64.whl基本用法
对于零样本/跨语言推理，请使用CosyVoice-300M模型。对于 sft 推理，请使用CosyVoice-300M-SFT模型。对于指令推理，请使用CosyVoice-300M-Instruct模型。首先，添加third_party/Matcha-TTS到您的PYTHONPATH。

开始网页操作
# change iic/CosyVoice-300M-SFT for sft inference, or iic/CosyVoice-300M-Instruct for instruct inference
python3 webui.py --port 50002--model_dir pretrained_models/CosyVoice-300M-Instruct 

我们在合成文本中加入需要润色的文本字段和需要的情感。

推理中：

推理结束，听听效果：

    还行，整体的情感润色起码有了，不是呆板的语音朗读。也算是自然语言板块的进步了。