基于pytorch框架实现NNLM代码

最新推荐文章于 2024-06-11 17:52:15 发布

hUaleeF

最新推荐文章于 2024-06-11 17:52:15 发布

阅读量918

点赞数

分类专栏： NLP Learning Notes 文章标签： 1024程序员节

本文链接：https://blog.csdn.net/hua_453/article/details/127498173

版权

基于pytorch框架实现NNLM代码

import torch
import torch.nn as nn
import torch.optim as optim
import torch.utils.data as Data
from torch.autograd import Variable
dtype = torch.FloatTensor

sentences = ["i like dog", "i love coffee", "i hate milk"]  # 句子数据集
n_steps = 2  # 用前几个单词来预测下一个单词，e.g. 2个
n_hidden = 2  # 隐藏层的节点个数，e.g. 2个
m = 2  # 词向量的长度

word_list = " ".join(sentences).split(" ")  # 获取所有的单词
print("未去重词表：", word_list)
word_list = list(set(word_list))  # 去重
print("去重词表：", word_list)
word_dict = {
   w: i for i, w in enumerate(word_list)}  # 单词->索引
print("单词索引：", word_dict)
number_dict = {
   i: w for i, w in enumerate(word_list)}  # 索引->单词
print("索引单词：", number_dict)
num_words = len