深度学习系列63：常用tts

1. espeak

mac上brew install，linux上apt-get install即可
使用也很简单：espeak -v zh 天气不错
跑的虽然快，但是音色特别“机械”

2. edge-tts

使用edge-tts（微软家的，需要联网）的示例代码如下：

import edge_tts,os,asyncio,sys
from pydub import AudioSegment,playback
async def read(text):
    tts = edge_tts.Communicate(text=text, voice='zh-CN-YunxiNeural',rate = '+5%')
    if 'temp.mp3' in os.listdir('.'):
    	os.system("rm temp.mp3")
    await tts.save(text+".mp3")

asyncio.run(read(sys.argv[1]))
playback.play(AudioSegment.from_mp3(sys.argv[1]+'.mp3'))

3. parler-tts

huggingface最近出的，可以通过语音提示 (voice prompts)，控制说话者的声调、语速、性别、噪音程度、情绪特征等。
目前只支持英文，期待后面支持中文。
如果想利用Mac上的mps，需要安装torch nightly。安装命令：
pip3 install --pre torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/nightly/cpu
测试代码：

# demo.py
import torch
from parler_tts import ParlerTTSForConditionalGeneration
from transformers import AutoTokenizer
import soundfile as sf

device = "cuda:0" if torch.cuda.is_available() else "cpu"

# 指定下载的模型的地址
model_path = f"{前面下载好的模型存放的目录}/parler-tts/parler_tts_mini_v0.1"

model = ParlerTTSForConditionalGeneration.from_pretrained(model_path).to(device)
tokenizer = AutoTokenizer.from_pretrained(model_path")

# 需要转成声音的文本
prompt = "Hey, how are you doing today?"
# 对声音的要求 
description = "A female speaker with a slightly low-pitched voice delivers her words quite expressively, in a very confined sounding environment with clear audio quality. She speaks very fast."
#description = "an old man"

input_ids = tokenizer(description, return_tensors="pt").input_ids.to(device)
prompt_input_ids = tokenizer(prompt, return_tensors="pt").input_ids.to(device)

generation = model.generate(input_ids=input_ids, prompt_input_ids=prompt_input_ids)
audio_arr = generation.cpu().numpy().squeeze()
sf.write("parler_tts_out.wav", audio_arr, model.config.sampling_rate)

4. sherpa

参考文档：https://k2-fsa.github.io/sherpa/sherpa.pdf
使用sherpa-onnx的参考代码如下，模型下载地址见https://hf-mirror.com/csukuangfj/vits-zh-aishell3

import soundfile as sf
import sherpa_onnx
def write(text,output_filename,sid=10,provider='cpu'):
    tts_config = sherpa_onnx.OfflineTtsConfig(
        model=sherpa_onnx.OfflineTtsModelConfig(
            vits=sherpa_onnx.OfflineTtsVitsModelConfig(
                model='vits-aishell3.onnx',
                lexicon='lexicon.txt',
                tokens='tokens.txt',
            ),
            provider=provider
        ),
        rule_fsts='number.fst',
        max_num_sentences=2,
    )
    audio = sherpa_onnx.OfflineTts(tts_config).generate(text, sid=sid)
    sf.write(
        output_filename,
        audio.samples,
        samplerate=audio.sample_rate,
        subtype="PCM_16",
    )

在GPU上不能直接用pip安装。首先安装好onnxruntime-gpu(jetson上参考https://www.elinux.org/Jetson_Zoo)，然后找到libonnxruntime_providers_***.so文件，创建软连接：
sudo ln -s /usr/local/libXXX.so /usr/lib/libXXX.so
然后参照https://k2-fsa.github.io/sherpa/onnx/python/index.html进行安装：
执行下面的命令

git clone https://github.com/k2-fsa/sherpa-onnx
export SHERPA_ONNX_CMAKE_ARGS="-DSHERPA_ONNX_ENABLE_GPU=ON"
cd sherpa-onnx
python3 setup.py install

5. pytts3和pytts4

pip install pyttsx3
pip install -U pyobjc

import pyttsx4
engine = pyttsx4.init()
engine.say('this is an english text to voice test.')
engine.runAndWait()

使用很简单

import pyttsx3
pyttsx3.speak("I will speak this text")

import pyttsx4
engine = pyttsx4.init()
engine.say('this is an english text to voice test.')
engine.runAndWait()

6. Melo-tts

参见https://github.com/myshell-ai/MeloTTS/blob/main/docs/install.md#linux-and-macos-install
速度很快，效果也不错。如果运行不起来，可以先检查一下pytorch版本。

git clone https://hub.nuaa.cf/myshell-ai/MeloTTS.git
cd MeloTTS
pip install -e .

然后安装unidic和unidic-lite，再将unidic-lite文件夹下的dicdata拷贝到unidic目录下
然后在~目录下安装nltk_data：https://hub.nuaa.cf/nltk/nltk_data
然后export HF_ENDPOINT=https://hf-mirror.com
然后执行melo-ui，或者直接在目录下执行python melo/app.py，可以打开网页：

命令行方式：melo “text-to-speech 领域近年来发展迅速” zh.wav -l ZH
使用python api：

from melo.api import TTS
model = TTS(language='ZH',device='auto')
model.tts_to_file("text-to-speech 领域近年来发展迅速", 1, 'zh.wav')

7. 其他

支持多语言，针对树莓派做了优化的https://github.com/rhasspy/piper#running-in-python
支持多语言，现在很流行的https://github.com/coqui-ai/TTS