【PaperReading】TERA: Self-Supervised Learning of Transformer Encoder Representation for Speech

最新推荐文章于 2024-08-16 07:57:16 发布

Talon不会写代码

最新推荐文章于 2024-08-16 07:57:16 发布

阅读量455

点赞数

分类专栏： Paper 文章标签： paper tera

本文链接：https://blog.csdn.net/talonzhang/article/details/109293911

版权

TERA是一种自监督语音预训练方法，通过多目标辅助任务在大量未标记的语音上预训练Transformer编码器，提高了表面特征的表现，并在多个任务中展现出强性能。该方法包括时间、通道和幅度三种辅助目标，以重建它们来学习更丰富的上下文表示。

摘要由CSDN通过智能技术生成

What’s main claim? Key idea?

This paper introduces a self-supervised speech pre-training method called TERA. The authors use a multi-target auxiliary task to pre-train Transformer Encoders on a large amount of unlabeled speech. And TERA achieved strong performance on many tasks by improving upon surface features.
Is there code available? Data?

code: https://github.com/andi611/Self-Supervised-Speech-Pretraining-and-Representation-Learning

data: LibriSpeech and TIMIT
Is the idea neat? Is it counter-intuitive?

I think it’s a neat idea. Self-supervised learning has emerged as an attractive approach to leverage knowledge from a large amount of unlabeled data. This paper i