李沐58_深层循环神经网络——自学笔记

!pip install --upgrade d2l==0.17.5  #d2l需要更新

简洁实现

实现多层循环神经网络所需的许多逻辑细节在高级API中都是现成的。

import torch
from torch import nn
from d2l import torch as d2l

batch_size, num_steps = 32, 35
train_iter, vocab = d2l.load_data_time_machine(batch_size, num_steps) #加载数据集

Downloading ../data/timemachine.txt from http://d2l-data.s3-accelerate.amazonaws.com/timemachine.txt...

因为我们有不同的词元，所以输入和输出都选择相同数量，即vocab_size。 隐藏单元的数量仍然是256
。 唯一的区别是，我们现在通过num_layers的值来设定隐藏层数。

vocab_size, num_hiddens, num_layers = len(vocab), 256, 2
num_inputs = vocab_size
device = d2l.try_gpu()
lstm_layer = nn.LSTM(num_inputs, num_hiddens, num_layers)
model = d2l.RNNModel(lstm_layer, len(vocab))
model = model.to(device)

训练和预测

num_epochs, lr = 500, 2
d2l.train_ch8(model, train_iter, vocab, lr*1.0, num_epochs, device)

perplexity 1.0, 243039.6 tokens/sec on cuda:0
time travelleryou can show black is white by argument said filby
travelleryou can show black is white by argument said filby

总结：

深度循环神经网络使用多个隐藏层来获得更多的非线性