Tensorflow：基于LSTM生成藏头诗

最新推荐文章于 2024-06-25 01:10:41 发布

log16

最新推荐文章于 2024-06-25 01:10:41 发布

阅读量1.5w

点赞数 7

文章标签：藏头诗 LSTM TensorFlow 古诗词

本文链接：https://blog.csdn.net/u014232627/article/details/71189078

版权

Tensorflow：基于LSTM生成藏头诗

最近在学习TensorFlow，学习到了RNN这一块，相关的资料不是很多，了解到使用RNN可以生成藏头诗之后，我就决定拿这个下手啦！
本文不介绍RNN以及LSTM的相关基本知识，如要了解，请自行百度。
本文是在学习了 TensorFlow7: 基于RNN生成古诗词
这一篇博客之后继续在其基础上修改的代码，若要了解相关内容可以先去上面的博客去看一下。

【注：本博客所使用的数据、代码、模型文件均已放在百度云上：
链接: https://pan.baidu.com/s/1qY4mt1y 密码: 47y2】

模型生成

首先我们要训练好模型。这里采用的是2层的LSTM框架，每层有128个隐藏层节点，batch_size设为64。训练数据来源于全唐诗（可在上面百度云资源分享当中找到）。特别注意到的一点是这里每训练完一次就对训练数据做shuffle。
源代码如下：

#!/usr/bin/python3
#-*- coding: UTF-8 -*-
import collections  
import numpy as np  
import tensorflow as tf  

#-------------------------------数据预处理---------------------------#  

poetry_file ='poetry.txt'  

# 诗集  
poetrys = []  
with open(poetry_file, "r") as f:  
    for line in f:  
        try:  
            line = line.decode('UTF-8')
            line = line.strip(u'\n')
            title, content = line.strip(u' ').split(u':')  
            content = content.replace(u' ',u'')  
            if u'_' in content or u'(' in content or u'（' in content or u'《' in content or u'[' in content:  
                continue  
            if len(content) < 5 or len(content) > 79:  
                continue  
            content = u'[' + content + u']'  
            poetrys.append(content)  
        except Exception as e:   
            pass  

# 按诗的字数排序  
poetrys = sorted(poetrys,key=lambda line: len(line))  
print('唐诗总数: ', len(poetrys))  

# 统计每个字出现次数  
all_words = []  
for poetry in poetrys:  
    all_words += [word for word in poetry]  
counter = collections.Counter(all_words)  
count_pairs = sorted(counter.items(), key=lambda x: -x[1])  
words, _ = zip(*count_pairs)  

# 取前多少个常用字  
words = words[:len(words)] + (' ',)  
# 每个字映射为一个数字ID  
word_num_map = dict(zip(words, range(len(words))))  
# 把诗转换为向量形式，参考TensorFlow练习1  
to_num = lambda word: word_num_map.get(word, len(words))  
poetrys_vector = [ list(map(to_num, poetry)) for poetry in poetrys]  
#[[314, 3199, 367, 1556, 26, 179, 680, 0, 3199, 41, 506, 40, 151, 4, 98, 1],  
#[339, 3, 133, 31, 302, 653, 512, 0, 37, 148, 294, 25, 54, 833, 3, 1, 965, 1315, 377, 1700, 562, 21, 37, 0, 2, 1253, 21, 36, 264, 877, 809, 1]  
#....]  

# 每次取64首诗进行训练  
batch_size = 64
n_chunk = len(poetrys_vector) // batch_size  

class DataSet(object):
    def __init__(self,data_size):
        self._data_size = data_size
        self._epochs_completed = 0
        self._index_in_epoch = 0
        self._data_index = np.arange(data_size)

    def next_batch(self,batch_size):
        start = self._index_in_epoch
        if start + batch_size > self._data_size:
            np.random.shuffle(self._data_index)
            self._epochs_completed = self._epochs_completed + 1