本周完成了两个任务:寻找并确定了几个英语口语对话的数据集,文本转语音模型的跑通。
文本转语音模型选用了全球公认最佳的微软语音模型,该模型生成的语音非常流畅,发音标准,注重语气和连贯性,与口语训练助手的定位非常契合。
首先注册Azure账户,申请资源组,申请具体textToSpeech资源
具体的模型使用部署官方教程:
Text to speech quickstart - Speech service - Azure AI services | Microsoft Learn
# Copyright (c) Microsoft. All rights reserved.
# Licensed under the MIT license. See LICENSE.md file in the project root for full license information.
# <code>
import azure.cognitiveservices.speech as speechsdk
# Creates an instance of a speech config with specified subscription key and service region.
# Replace with your own subscription key and service region (e.g., "westus").
speech_key, service_region = "", ""
speech_config = speechsdk.SpeechConfig(subscription=speech_key, region=service_region)
speech_config.speech_synthesis_language = "en-US"
# Set the voice name, refer to https://aka.ms/speech/voices/neural for full list.
speech_config.speech_synthesis_voice_name = "en-US-AvaMultilingualNeural"
# Creates a speech synthesizer using the default speaker as audio output.
speech_synthesizer = speechsdk.SpeechSynthesizer(speech_config=speech_config)
# Receives a text from console input.
print("Type some text that you want to speak...")
text = input()
# Synthesizes the received text to speech.
# The synthesized speech is expected to be heard on the speaker with this line executed.
result = speech_synthesizer.speak_text_async(text).get()
print(result)
# Checks result.
if result.reason == speechsdk.ResultReason.SynthesizingAudioCompleted:
print("Speech synthesized to speaker for text [{}]".format(text))
elif result.reason == speechsdk.ResultReason.Canceled:
cancellation_details = result.cancellation_details
print("Speech synthesis canceled: {}".format(cancellation_details.reason))
if cancellation_details.reason == speechsdk.CancellationReason.Error:
if cancellation_details.error_details:
print("Error details: {}".format(cancellation_details.error_details))
print("Did you update the subscription info?")
# </code>
运行之后根据输入的文本,生成模型语音通过设备扬声器放出,还有一种是通过返回in-memory stream的方式:
speech_synthesizer = speechsdk.SpeechSynthesizer(speech_config=speech_config, audio_config=None)
result = speech_synthesizer.speak_text_async("I'm excited to try text to speech").get()
stream = AudioDataStream(result)
微调数据集方面,目前寻找到了一个开源口语对话数据集Daily_dialog,在huggingface上可以直接下载该数据集:
DailyDialog是一个高质量的多回合开放域英语对话数据集。它包含13118个对话,分为11118个对话的训练集和1000个对话的验证和测试集。平均而言,每段对话大约有8个演讲者回合,每回合大约有15个标记。用python查看数据集的一行内容: