【AC】Accent Conversion论文整理

分为parallel data-based Accent Conversion 和non-parallel data-based Accent Conversion, 这里的parallel data指的是 source speaker 和 target speaker 说同样的文本的语料。

1. Parallel data based accent conversion methods

大部分方法依赖bottleneck feature / PPG features作为conversion的中间特征，bottleneck features/ PPG features作为耦合了accent和content的信息。[1][2][3]

• Foreign Accent Conversion by Synthesizing Speech from Phonetic Posteriorgrams(INTERSPEECH 2019, Zhao Guanlong)

L2 speech synthesizer负责建模timbre, acoustic model负责建模 accent 和 content. 训练时候应用L2 PPG重构L2音频，推理阶段应用L1 PPG驱动L2 speech synthesizer得到转换后的输出。Acoustic model 是kaldi预训练好的 DNN-based ASR声学模型，L2 speech synthesizer应用了tacotron结构。

• Converting Foreign Accent Speech Without a Reference(TASLP 2021, Zhao Guanlong)

先用 L2 BNF features 驱动 L2 synthesizer 生成 L2 的句子，使得 L2 synthesizer 完成timbre的建模。随后应用L1 BNF驱动 L2 synthesizer 生成 accent conversion 的 ground truth, 用于后续 Pronounciation correction 的训练。L2 synthesizer 和 Pronounciation correction model 的结构都是tacotron, 细节有所不同。

• Accentron: Foreign accent conversion to arbitrary non-native speakers using zero-shot learning(Computer Speech & Language 2022, Ding Shaojin)

通过source speech的BNF features和speaker embeddings, accent embedings重构target speech。Seq2seq Model 参考tacotron结构（p-Bi-LSTM）。BNF耦合了source的content和accent，通过从 target mel 中提取accent和speaker embedding，因此Seq2seq Model需要负责建模timbre和accent。

• Zero-Shot Foreign Accent Conversion without a Native Reference(INTERSPEECH 2022)

将accent conversion分成两个任务：convert accent 和 speech synthesize. Translator 负责将有口音的BNF转成没有口音的BNF，Synthesizer负责将没有口音的BNF生成mel(speaker是source speaker的)。Translator和Synthesizer都用了tacotron结构。

[1] Foreign Accent Conversion by Synthesizing Speech from Phonetic Posteriorgrams(INTERSPEECH 2019)
[2] Accentron: Foreign accent conversion to arbitrary non-native speakers using zero-shot learning(Computer Speech & Language 2022, Ding Shaojin)
[3] Zero-Shot Foreign Accent Conversion without a Native Reference(INTERSPEECH 2022)