[Pytorch]pytorch中的LSTM模型

版权声明:本文为博主原创文章,遵循 CC 4.0 by-sa 版权协议,转载请附上原文出处链接和本声明。
本文链接:https://blog.csdn.net/android_ruben/article/details/80206792

公式表示

Pytorch中LSTM的公式表示为:

it=σ(Wiixt+bii+Whih(t1)+bhi)

ft=σ(Wifxt+bif+Whfh(t1)+bhf)

gt=tanh(Wigxt+big+Whgh(t1)+bhg)

ot=σ(Wioxt+bio+Whoh(t1)+bho)

ct=ftc(t1)+itgt

ht=ottanh(ct)

其中it为输入门,ft为遗忘门,gt为细胞门的输出,ot为输出门,ct为细胞t时刻的状态,htt时刻的隐藏层状态。

定义

Pytorch中LSTM的定义如下:

class torch.nn.LSTM(*args, **kwargs)

参数列表

  • input_size:x的特征维度
  • hidden_size:隐藏层的特征维度
  • num_layers:lstm隐层的层数,默认为1
  • bias:False则bih=0和bhh=0. 默认为True
  • batch_first:True则输入输出的数据格式为 (batch, seq, feature)
  • dropout:除最后一层,每一层的输出都进行dropout,默认为: 0
  • bidirectional:True则为双向lstm默认为False
  • 输入:input, (h0, c0)
  • 输出:output, (hn,cn)

输入数据格式:
input(seq_len, batch, input_size)
h0(num_layers * num_directions, batch, hidden_size)
c0(num_layers * num_directions, batch, hidden_size)

输出数据格式:
output(seq_len, batch, hidden_size * num_directions)
hn(num_layers * num_directions, batch, hidden_size)
cn(num_layers * num_directions, batch, hidden_size)

实例:基于LSTM的词性标注模型

import torch
import gensim
torch.manual_seed(2)

datas=[('你 叫 什么 名字 ?','n v n n f'),('今天 天气 怎么样 ?','n n adj f'),]
words=[ data[0].split() for data in datas]
tags=[ data[1].split() for data in datas]


id2word=gensim.corpora.Dictionary(words)
word2id=id2word.token2id

id2tag=gensim.corpora.Dictionary(tags)
tag2id=id2tag.token2id

def sen2id(inputs):
    return [word2id[word] for word in inputs]
def tags2id(inputs):
    return [tag2id[word] for word in inputs]
# print(sen2id('你 叫 什么 名字'.split()))
def formart_input(inputs):
    return torch.autograd.Variable(torch.LongTensor(sen2id(inputs)))
def formart_tag(inputs):
    return torch.autograd.Variable(torch.LongTensor(tags2id(inputs)),)

class LSTMTagger(torch.nn.Module):
    def __init__(self,embedding_dim,hidden_dim,voacb_size,target_size):
        super(LSTMTagger,self).__init__()
        self.embedding_dim=embedding_dim
        self.hidden_dim=hidden_dim
        self.voacb_size=voacb_size
        self.target_size=target_size
        self.lstm=torch.nn.LSTM(self.embedding_dim,self.hidden_dim)
        self.log_softmax=torch.nn.LogSoftmax()
        self.embedding=torch.nn.Embedding(self.voacb_size,self.embedding_dim)
        self.hidden=(torch.autograd.Variable(torch.zeros(1,1,self.hidden_dim)),torch.autograd.Variable(torch.zeros(1,1,self.hidden_dim)))
        self.out2tag=torch.nn.Linear(self.hidden_dim,self.target_size)
    def forward(self,inputs):
        input=self.embedding((inputs))
        out,self.hidden=self.lstm(input.view(-1,1,self.embedding_dim),self.hidden)
        tags=self.log_softmax(self.out2tag(out.view(-1,self.hidden_dim)))
        return tags

model=LSTMTagger(3,3,len(word2id),len(tag2id))
loss_function=torch.nn.NLLLoss()
optimizer=torch.optim.SGD(model.parameters(),lr=0.1)
for _ in range(100):
    model.zero_grad()
    input=formart_input('你 叫 什么 名字'.split())
    tags=formart_tag('n n adj f'.split())
    out=model(input)
    loss=loss_function(out,tags)
    loss.backward(retain_variables=True)
    optimizer.step()
    print(loss.data[0])
input=formart_input('你 叫 什么'.split())
out=model(input)
out=torch.max(out,1)[1]
print([id2tag[out.data[i]] for i in range(0,out.size()[0])])
展开阅读全文

没有更多推荐了,返回首页