《PyTorch深度学习实践》第十二课（循环神经网络RNN）附加Embedding

b站刘二视频，地址：

《PyTorch深度学习实践》完结合集_哔哩哔哩_bilibili

独热向量的冗余过多，因此采用更为稠密的Embedding层 

 假设输入向量为四维（代表每一个输入有四种可能性），构建一个矩阵，完成转化

 反向传播，直接用矩阵乘法的导数就可以

最终整体网络

 Embedding层的参数

num_embeddings就是独热向量的维度

embedding_dim就是前面那个矩阵的列数

输入必须是longTensor

 改造后模型代码

输入输出维度

 

import torch

input_size = 4
hidden_size = 8
batch_size = 1
num_layers = 2
num_class = 4
embedding_size = 10
seq_len = 5


idx2char = ['e', 'h', 'l', '0']
x_data = [[1, 0, 2, 2, 3]] #(batch, seqlen)
y_data = [3, 1, 2, 3, 2] #(batch * seqlen)

inputs = torch.LongTensor(x_data)
labels = torch.LongTensor(y_data)

class module(torch.nn.Module):
    def __init__(self):
        super(module, self).__init__()

        self.emb = torch.nn.Embedding(input_size, embedding_size)
        self.rnn = torch.nn.RNN(input_size=embedding_size, hidden_size=hidden_size,
                                num_layers=num_layers, batch_first=True)
        self.fc = torch.nn.Linear(hidden_size, num_class)

    def forward(self, x):
        hidden = torch.zeros(num_layers, x.size(0), hidden_size)
        x = self.emb(x)
        x,_ = self.rnn(x, hidden)
        x = self.fc(x)
        return x.view(-1, num_class)

net = module()
criterion = torch.nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(net.parameters(), lr=0.05)

for epoch in range(15):
    optimizer.zero_grad()
    outputs = net(inputs)
    loss = criterion(outputs, labels)
    loss.backward()
    optimizer.step()

    _, idx = outputs.max(dim=1)
    idx = idx.data.numpy()
    print('Predicted: ', ''.join([idx2char[x] for x in idx]), end='')
    print(', Epoch [%d / 15] loss = %.3f'%(epoch + 1, loss.item()))