Paraformer解读（1）基于self-attention和dfsmn的encoder

Pika在线

于 2024-06-05 00:41:21 发布

阅读量429

点赞数 2

分类专栏： AI语音文章标签： python 深度学习 paraformer

本文链接：https://blog.csdn.net/Ephemeroptera/article/details/139457093

版权

DFSMN

dfsmn

SAN-M

san-m

python实现

import torch
import torch.nn as nn
import torch.nn.functional as F

class PositionalEncoding(nn.Module):
    def __init__(self, d_model, dropout=0.1, max_len=5000):
        super(PositionalEncoding, self).__init__()
        self.dropout = nn.Dropout(p=dropout)

        position = torch.arange(max_len).unsqueeze(1)
        div_term = torch.exp(torch.arange(0, d_model, 2) * -(torch.log(torch.tensor(10000.0)) / d_model))
        pe = torch.zeros(max_len, 1, d_model)
        pe[:, 0, 0::2] = torch.sin(position * div_term)
        pe[:, 0, 1::2