import torch
if torch.cuda.is_available():# Tell PyTorch to use the GPU.
device = torch.device("cuda")print('There are %d GPU(s) available.'% torch.cuda.device_count())print('We will use the GPU:', torch.cuda.get_device_name(0))else:print('No GPU available, using the CPU instead.')
device = torch.device("cpu")
There are 1 GPU(s) available.
We will use the GPU: GeForce GTX 1070
设置参数
import torchtext
from torchtext.vocab import Vectors
import numpy as np
import random
# 为了保证实验结果可以复现,我们经常会把各种random seed固定在某一个值
random.seed(53113)
np.random.seed(53113)
torch.manual_seed(53113)
BATCH_SIZE =32#一个batch多少个句子
EMBEDDING_SIZE =650#每个单词多少维
MAX_VOCAB_SIZE =50000#单词总数
下载数据集
TEXT = torchtext.data.Field(lower=True)# https://s0pytorch0org.icopy.site/text/data.html?highlight=torchtext%20data%20field#torchtext.data.Field# torchtext提供了LanguageModelingDataset这个class来帮助我们处理语言模型数据集
train, val, test = torchtext.datasets.LanguageModelingDataset.splits(
path=".",
train="train.txt",
validation="dev.txt",
test="test.txt",
text_field=TEXT)
# 如果一直往后传,会造成整个计算图很庞大,反向传播会内存崩溃。所有每次一个batch的计# 算图迭代完成后,需要把计算图截断,只保留隐藏层的输出值。defrepackage_hidden(h):"""Wraps hidden states in new Tensors, to detach them from their history."""ifisinstance(h, torch.Tensor):# 这个是GRU的截断,因为只有一个隐藏层# 判断h是不是torch.Tensorreturn h.detach()# 截断计算图,h是全的计算图的开始,只是保留了h的值else:# 这个是LSTM的截断,有两个隐藏层,格式是元组returntuple(repackage_hidden(v)for v in h)
epoch 0 iter 0 loss 10.730563163757324
best model, val loss: 10.478901235690277
D:\Anaconda\envs\jianbo\lib\site-packages\torch\serialization.py:402: UserWarning: Couldn't retrieve source code for container of type My_Model. It won't be checked for correctness upon loading.
"type " + obj.__name__ + ". It won't be checked "
epoch 0 iter 1000 loss 6.0242919921875
epoch 0 iter 2000 loss 6.029582500457764
epoch 0 iter 3000 loss 5.8461594581604
epoch 0 iter 4000 loss 5.5147223472595215
epoch 0 iter 5000 loss 5.937921047210693
epoch 0 iter 6000 loss 5.6236090660095215
epoch 0 iter 7000 loss 5.482613563537598
epoch 0 iter 8000 loss 5.344069004058838
epoch 0 iter 9000 loss 5.418025970458984
epoch 1 iter 0 loss 5.486691474914551
best model, val loss: 5.002634433592716
epoch 1 iter 1000 loss 5.0923237800598145
epoch 1 iter 2000 loss 5.381066799163818
epoch 1 iter 3000 loss 5.237982273101807
epoch 1 iter 4000 loss 4.973425388336182
epoch 1 iter 5000 loss 5.4851861000061035
epoch 1 iter 6000 loss 5.201869010925293
epoch 1 iter 7000 loss 5.1173810958862305
epoch 1 iter 8000 loss 5.007303237915039
epoch 1 iter 9000 loss 5.120178699493408
who become born epicurus and looking for them as a <unk> is to print using hypocrisy that married his corresponding a buffer of his bicycle and put her came that <unk> into the drink the abuse of manganese s into the liver and prayers the second it is his own nowhere of the earth recognizes his origin but has primarily been used by arthur gardner largely written for this reason he differs from his eight sequel to the catherine copula which appears to be that of it encoding beethoven s demonstration the last ship desires to invent wittenberg was an