seq2seq模型简单实现

最新推荐文章于 2024-08-22 08:42:14 发布
最新推荐文章于 2024-08-22 08:42:14 发布
阅读量2.5k 收藏 1
点赞数
分类专栏： 机器学习 python编程
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/luoyexuge/article/details/78904471
版权

理论参考:https://mp.weixin.qq.com/s/qXMRHxDDRa-_rJZMhXWB4w

http://blog.csdn.net/liuchonge/article/details/78824572

http://blog.csdn.net/liuchonge/article/details/78856692




代码参考 https://github.com/NELSONZHAO/zhihu/blob/master/basic_seq2seq/Seq2seq_char.ipynb


不过作者使用的tensorflow1.1的版本，我用的是tensorflow1.3的版本，在返回结果的时候稍微有点不同，数据都来自上面的博客


  
  
  1. In [1]: import tensorflow as tf
  3. In [2]: tf.__version__
  4. Out[2]: '1.3.0'

先看下data_utils.py

   
   
  1. import numpy as np
  2. source_path="data/letters_source.txt"
  3. target_path="data/letters_target.txt"
  4. def  read_file(source_path,target_path):
  5.     with open(source_path,'r',encoding="utf-8") as  f:
  6.         source_data=f.read()
  7.     with  open(target_path,'r',encoding='utf-8') as f:
  8.         target_data = f.read()
  9.     return source_data,target_data
  11. def extract_character_vocab(data):
  12.     """
  13.     构造映射表
  14.     :param data:
  15.     :return:
  16.     """
  17.     special_words=['<PAD>','<UNK>','<GO>','<EOS>']
  18.     set_words=list(set([ character for line in data.split('\n') for character in line]))
  19.     int_to_vocab={idx:word for idx,word in enumerate(special_words+set_words)}
  20.     vocab_to_int={word:idx for idx ,word in int_to_vocab.items()}
  21.     return int_to_vocab,vocab_to_int
  24. def file_to_ids(source_path="data/letters_source.txt",target_path="data/letters_target.txt"):
  25.     source_data, target_data=read_file(source_path,target_path)
  26.     source_int_to_letter, source_letter_to_int = extract_character_vocab(source_data)
  27.     target_int_to_letter, target_letter_to_int = extract_character_vocab(target_data)
  28.     source_int=[[ source_letter_to_int.get(letter,source_letter_to_int['<UNK>'])
  29.                    for letter in line  ]for line in source_data.split('\n')]
  31.     target_int=[ [target_letter_to_int.get(letter,target_letter_to_int['<UNK>'])
  32.                   for letter in line] +[target_letter_to_int['<EOS>']] for line in target_data.split('\n')]
  35.     return source_int,target_int,source_letter_to_int,target_letter_to_int
  38. def pad_sentence_batch(sentence_batch, pad_int):
  39.     '''
  40.     对batch中的序列进行补全，保证batch中的每行都有相同的sequence_length
  42.     参数：
  43.     - sentence batch
  44.     - pad_int: <PAD>对应索引号
  45.     '''
  46.     max_sentence = max([len(sentence) for sentence in sentence_batch])
  47.     return [sentence + [pad_int] * (max_sentence - len(sentence)) for sentence in sentence_batch]
  50. def get_batches(targets, sources, batch_size, source_pad_int, target_pad_int):
  51.     '''
  52.     定义生成器，用来获取batch
  53.     '''
  54.     for batch_i in range(0, len(sources) // batch_size):
  55.         start_i = batch_i * batch_size
  56.         sources_batch = sources[start_i:start_i + batch_size]
  57.         targets_batch = targets[start_i:start_i + batch_size]
  58.         # 补全序列
  59.         pad_sources_batch = np.array(pad_sentence_batch(sources_batch, source_pad_int))
  60.         pad_targets_batch = np.array(pad_sentence_batch(targets_batch, target_pad_int))
  62.         # 记录每条记录的长度
  63.         targets_lengths = []
  64.         for target in targets_batch:
  65.             targets_lengths.append(len(target))
  67.         source_lengths = []
  68.         for source in sources_batch:
  69.             source_lengths.append(len(source))
  71.         yield pad_targets_batch, pad_sources_batch, targets_lengths, source_lengths
  73. if __name__=='__main__':
  74.     source_data, target_data=read_file(source_path,target_path)
  75.     print(source_data.split("\n")[0:10])
  76.     print(target_data.split("\n")[0:10])
  77.     #{0: '<PAD>', 1: '<UNK>', 2: '<GO>',
  78.     # 3: '<EOS>', 4: 'v', 5: 'y', 6: 'i', 7: 'e', 8: 'd', 9:
  79.     source_int, target_int,_,_=file_to_ids()
  80.     print(source_int[:10])
  81.     print(target_int[0:10])

config.py:

   
   
  1. class Config:
  2.     batch_size = 128   # Batch Size
  3.     rnn_size = 50      # RNN Size
  4.     num_layers = 2     # Number of Layers
  5.     encoding_embedding_size = 15 # Embedding Size
  6.     decoding_embedding_size = 15  # Embedding Size
  7.     learning_rate = 0.001   # Learning Rate
  8.     epochs=100 #模型每隔60步骤打印出loss

model.py

   
   
  1. import tensorflow as tf
  2. from tensorflow.python.layers.core import Dense
  4. class Seq2SeqModelBase(object):
  5.     def __init__(self,config,source_letter_to_int,target_letter_to_int):
  6.         self.config=config
  7.         self.learing_rate=self.config.learning_rate
  8.         self.batch_size=self.config.batch_size
  9.         self.encoding_embedding_size=self.config.encoding_embedding_size
  10.         self.decoding_embedding_size=self.config.decoding_embedding_size
  11.         self.rnn_size=self.config.rnn_size
  12.         self.num_layers=self.config.num_layers
  13.         self.source_letter_to_int=source_letter_to_int
  14.         self.target_letter_to_int=target_letter_to_int
  15.         self.inputs=tf.placeholder(tf.int32,[None,None],name='inputs')
  16.         self.targets=tf.placeholder(tf.int32,[None,None],name='target')
  17.         # 定义target序列最大长度
  18.         # （之后target_sequence_length和
  19.         # source_sequence_length会作为feed_dict的参数）
  20.         self.target_sequence_length=tf.placeholder(tf.int32, shape=[None],name='target_sequence_length')
  21.         self.max_target_sequence_length=tf.reduce_max(self.target_sequence_length,name='max_target_len')
  22.         self.source_sequence_length = tf.placeholder(tf.int32, shape=[None], name='source_sequence_length')
  23.         self.train()
  24.     #encode层
  25.     def get_encoder_layer(self,input_data, rnn_size, num_layers,
  26.                           source_sequence_length, source_vocab_size,
  27.                           encoding_embedding_size):
  28.         '''
  29.         构造Encoder层
  31.         参数说明：
  32.         - input_data: 输入tensor
  33.         - rnn_size: rnn隐层结点数量
  34.         - num_layers: 堆叠的rnn cell数量
  35.         - source_sequence_length: 源数据的序列长度
  36.         - source_vocab_size: 源数据的词典大小
  37.         - encoding_embedding_size: embedding的大小
  38.         '''
  39.         # Encoder embedding
  40.         encoder_embed_input = tf.contrib.layers.embed_sequence(input_data, source_vocab_size,
  41.                                                                encoding_embedding_size)
  43.         # RNN cell
  44.         def get_lstm_cell(rnn_size):
  45.             lstm_cell = tf.contrib.rnn.LSTMCell(rnn_size,
  46.                                                 initializer=tf.random_uniform_initializer(-0.1, 0.1, seed=2))
  47.             return lstm_cell
  49.         cell = tf.contrib.rnn.MultiRNNCell([get_lstm_cell(rnn_size) for _ in range(num_layers)])
  51.         encoder_output, encoder_state = tf.nn.dynamic_rnn(cell, encoder_embed_input,
  52.                                                           sequence_length=source_sequence_length, dtype=tf.float32)
  54.         return encoder_output, encoder_state
  55.      # encode层 对target数据进行预处理
  57.     def process_decoder_input(self,data, vocab_to_int, batch_size):
  58.         '''
  59.         补充<GO>，并移除最后一个字符
  60.         '''
  61.         # cut掉最后一个字符
  62.         ending = tf.strided_slice(data, [0, 0], [batch_size, -1], [1, 1])
  63.         decoder_input = tf.concat([tf.fill([batch_size, 1], vocab_to_int['<GO>']), ending], 1)
  65.         return decoder_input
  67.     """
  68.     对数据进行embedding
  69.         同样地，我们还需要对target数据进行embedding，使得它们能够传入Decoder中的RNN
  70.     """
  72.     def decoding_layer(self,target_letter_to_int, decoding_embedding_size, num_layers, rnn_size,
  73.                        target_sequence_length, max_target_sequence_length, encoder_state, decoder_input):
  74.         '''
  75.         构造Decoder层
  77.         参数：
  78.         - target_letter_to_int: target数据的映射表
  79.         - decoding_embedding_size: embed向量大小
  80.         - num_layers: 堆叠的RNN单元数量
  81.         - rnn_size: RNN单元的隐层结点数量
  82.         - target_sequence_length: target数据序列长度
  83.         - max_target_sequence_length: target数据序列最大长度
  84.         - encoder_state: encoder端编码的状态向量
  85.         - decoder_input: decoder端输入
  86.         '''
  87.         # 1. Embedding
  88.         target_vocab_size = len(target_letter_to_int)
  89.         decoder_embeddings = tf.Variable(tf.random_uniform([target_vocab_size, decoding_embedding_size]))
  90.         decoder_embed_input = tf.nn.embedding_lookup(decoder_embeddings, decoder_input)
  92.         # 2. 构造Decoder中的RNN单元
  93.         def get_decoder_cell(rnn_size):
  94.             decoder_cell = tf.contrib.rnn.LSTMCell(rnn_size,
  95.                                                    initializer=tf.random_uniform_initializer(-0.1, 0.1, seed=2))
  96.             return decoder_cell
  98.         cell = tf.contrib.rnn.MultiRNNCell([get_decoder_cell(rnn_size) for _ in range(num_layers)])
  100.         # 3. Output全连接层
  101.         output_layer = Dense(target_vocab_size,
  102.                              kernel_initializer=tf.truncated_normal_initializer(mean=0.0, stddev=0.1))
  104.         # 4. Training decoder
  105.         with tf.variable_scope("decode"):
  106.             # 得到help对象
  107.             training_helper = tf.contrib.seq2seq.TrainingHelper(inputs=decoder_embed_input,
  108.                                                                 sequence_length=target_sequence_length)
  109.             # 构造decoder
  110.             training_decoder = tf.contrib.seq2seq.BasicDecoder(cell,
  111.                                                                training_helper,
  112.                                                                encoder_state,
  113.                                                                output_layer)
  115.             #training_decoder_output, _,_ = tf.contrib.seq2seq.dynamic_decode(training_decoder)
  116.             training_decoder_output, _,_ = tf.contrib.seq2seq.dynamic_decode(training_decoder,
  117.                                                                           impute_finished=True,
  118.                                                                           maximum_iterations=max_target_sequence_length)
  119.         # 5. Predicting decoder
  120.         # 与training共享参数
  121.         with tf.variable_scope("decode", reuse=True):
  122.             # 创建一个常量tensor并复制为batch_size的大小
  123.             start_tokens = tf.tile(tf.constant([target_letter_to_int['<GO>']], dtype=tf.int32), [self.batch_size],
  124.                                    name='start_tokens')
  125.             predicting_helper = tf.contrib.seq2seq.GreedyEmbeddingHelper(decoder_embeddings,
  126.                                                                          start_tokens,
  127.                                                                          target_letter_to_int['<EOS>'])
  128.             predicting_decoder = tf.contrib.seq2seq.BasicDecoder(cell,
  129.                                                                  predicting_helper,
  130.                                                                  encoder_state,
  131.                                                                  output_layer)
  132.             predicting_decoder_output, _ ,_= tf.contrib.seq2seq.dynamic_decode(predicting_decoder,
  133.                                                                              impute_finished=True,
  134.                                                                              maximum_iterations=max_target_sequence_length)
  136.         return training_decoder_output, predicting_decoder_output
  138.     def _get_simple_lstm(self, rnn_size, layer_size):
  139.         lstm_layers = [tf.contrib.rnn.LSTMCell(rnn_size) for _ in range(layer_size)]
  140.         return tf.contrib.rnn.MultiRNNCell(lstm_layers)
  142.     """
  143.     Seq2Seq
  144.     上面已经构建完成Encoder和Decoder，下面将这两部分连接起来，构建seq2seq模型
  145.     """
  146.     def seq2seq_model(self,input_data, targets, lr, target_sequence_length,
  147.                       max_target_sequence_length, source_sequence_length,
  148.                       source_vocab_size, target_vocab_size,
  149.                       encoder_embedding_size, decoder_embedding_size,
  150.                       rnn_size, num_layers):
  151.         # 获取encoder的状态输出
  152.         _, encoder_state = self.get_encoder_layer(input_data,
  153.                                              rnn_size,
  154.                                              num_layers,
  155.                                              source_sequence_length,
  156.                                              source_vocab_size,
  157.                                                   encoder_embedding_size)
  159.         # 预处理后的decoder输入
  160.         decoder_input = self.process_decoder_input(targets, self.target_letter_to_int, self.batch_size)
  162.         # 将状态向量与输入传递给decoder
  163.         training_decoder_output, predicting_decoder_output = self.decoding_layer(self.target_letter_to_int,
  164.                                                                                  decoder_embedding_size,
  165.                                                                             num_layers,
  166.                                                                             rnn_size,
  167.                                                                             target_sequence_length,
  168.                                                                             max_target_sequence_length,
  169.                                                                             encoder_state,
  170.                                                                             decoder_input
  171.                                                                     )
  173.         return training_decoder_output, predicting_decoder_output
  174.     def train(self):
  175.         training_decoder_output, predicting_decoder_output = self.seq2seq_model(self.inputs,
  176.                                                                                 self.targets,
  177.                                                                                 self.learing_rate,
  178.                                                                                 self.target_sequence_length,
  179.                                                                                 self.max_target_sequence_length,
  180.                                                                                 self.source_sequence_length,
  181.                                                                            len(self.source_letter_to_int),
  182.                                                                            len(self.target_letter_to_int),
  183.                                                                                 self.encoding_embedding_size,
  184.                                                                                 self.decoding_embedding_size,
  185.                                                                                 self.rnn_size,
  186.                                                                                 self.num_layers)
  188.         training_logits = tf.identity(training_decoder_output.rnn_output, 'logits')
  189.         predicting_logits = tf.identity(predicting_decoder_output.sample_id, name='predictions')
  191.         masks = tf.sequence_mask(self.target_sequence_length, self.max_target_sequence_length, dtype=tf.float32, name='masks')
  192.         print(' training_logits:{}'.format(training_logits.get_shape()))
  193.         print(' self.targets:{}'.format(self.targets.get_shape()))
  194.         self.cost = tf.contrib.seq2seq.sequence_loss(
  195.             training_logits,
  196.             self.targets,
  197.             masks)
  199.         # targets = tf.reshape(self.targets, [-1])
  200.         # predicting_logits = tf.identity(predicting_decoder_output.sample_id, name='predictions')
  201.         # logits_flat = tf.reshape(training_decoder_output.rnn_output, [-1, len(self.target_letter_to_int)])
  202.         # print('shape logits_flat:{}'.format(logits_flat.shape))
  203.         # print('shape logits:{}'.format(training_decoder_output.rnn_output.shape))
  205.         #self.cost = tf.losses.sparse_softmax_cross_entropy(labels=targets, logits=logits_flat)
  207.         # Optimizer
  208.         optimizer = tf.train.AdamOptimizer(self.learing_rate)
  210.         # Gradient Clipping
  211.         gradients = optimizer.compute_gradients(self.cost)
  212.         capped_gradients = [(tf.clip_by_value(grad, -5., 5.), var) for grad, var in gradients if grad is not None]
  213.         self.train_op = optimizer.apply_gradients(capped_gradients)


run.py

   
   
  1. import tensorflow as tf
  2. from data_utils import file_to_ids,get_batches
  3. from configuration import  Config
  4. from model import  Seq2SeqModelBase
  6. config=Config()
  8. batch_size=config.batch_size
  9. source_int,target_int,source_letter_to_int,target_letter_to_int=file_to_ids()
  10. model=Seq2SeqModelBase(config,source_letter_to_int,target_letter_to_int)
  11. # 将数据集分割为train和validation
  12. train_source = source_int[batch_size:]
  13. train_target = target_int[batch_size:]
  14. # 留出一个batch进行验证
  15. valid_source = source_int[:batch_size]
  16. valid_target = target_int[:batch_size]
  17. (valid_targets_batch, valid_sources_batch, valid_targets_lengths, valid_sources_lengths) = next(get_batches(valid_target, valid_source, batch_size,
  18.                            source_letter_to_int['<PAD>'],
  19.                            target_letter_to_int['<PAD>']))
  21. display_step = 50  # 每隔50轮输出loss
  23. checkpoint = "modelsave/trained_model.ckpt"
  25. # 构造graph
  26. with tf.Session() as sess:
  27.     sess.run(tf.global_variables_initializer())
  29.     for epoch_i in range(1, config.epochs + 1):
  30.         for batch_i, (targets_batch, sources_batch, targets_lengths, sources_lengths) in enumerate(
  31.                 get_batches(train_target, train_source, batch_size,
  32.                             source_letter_to_int['<PAD>'],
  33.                             target_letter_to_int['<PAD>'])):
  35.             _, loss = sess.run(
  36.                 [model.train_op, model.cost],
  37.                 feed_dict={model.inputs: sources_batch,
  38.                  model.targets: targets_batch,
  39.                  model.target_sequence_length: targets_lengths,
  40.                  model.source_sequence_length: sources_lengths})
  42.             if batch_i % display_step == 0 and batch_i>0 :
  43.                 # 计算validation loss
  44.                 validation_loss = sess.run(
  45.                     [model.cost],
  46.                     feed_dict={model.inputs: valid_sources_batch,
  47.                      model.targets: valid_targets_batch,
  48.                      model.target_sequence_length: valid_targets_lengths,
  49.                      model.source_sequence_length: valid_sources_lengths})
  50.                 print('Epoch {:>3}/{} Batch {:>4}/{} - Training Loss: {:>6.3f}  '
  51.                       '- Validation loss: {:>6.3f}'.
  52.                       format(epoch_i,config.epochs,batch_i,len(train_source) // batch_size,loss,validation_loss[0]))
  54.     # 保存模型
  55.     saver = tf.train.Saver()
  56.     saver.save(sess, checkpoint)
  57.     print('Model Trained and Saved')

run结果:

   
   
  1. 2017-12-26 17:06:15.971564: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
  2. 2017-12-26 17:06:15.971732: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
  3. 2017-12-26 17:06:15.971741: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
  4. 2017-12-26 17:06:15.971747: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
  5. Epoch   1/100 Batch   50/77 - Training Loss:  2.885  - Validation loss:  2.884
  6. Epoch   2/100 Batch   50/77 - Training Loss:  2.137  - Validation loss:  2.146
  7. Epoch   3/100 Batch   50/77 - Training Loss:  1.783  - Validation loss:  1.801
  8. Epoch   4/100 Batch   50/77 - Training Loss:  1.528  - Validation loss:  1.537
  9. Epoch   5/100 Batch   50/77 - Training Loss:  1.320  - Validation loss:  1.307
  10. Epoch   6/100 Batch   50/77 - Training Loss:  1.123  - Validation loss:  1.111
  11. Epoch   7/100 Batch   50/77 - Training Loss:  0.961  - Validation loss:  0.952
  12. Epoch   8/100 Batch   50/77 - Training Loss:  0.827  - Validation loss:  0.826
  13. Epoch   9/100 Batch   50/77 - Training Loss:  0.726  - Validation loss:  0.733
  14. Epoch  10/100 Batch   50/77 - Training Loss:  0.643  - Validation loss:  0.642
  15. Epoch  11/100 Batch   50/77 - Training Loss:  0.576  - Validation loss:  0.563
  16. Epoch  12/100 Batch   50/77 - Training Loss:  0.518  - Validation loss:  0.492
  17. Epoch  13/100 Batch   50/77 - Training Loss:  0.460  - Validation loss:  0.435
  18. Epoch  14/100 Batch   50/77 - Training Loss:  0.401  - Validation loss:  0.382
  19. Epoch  15/100 Batch   50/77 - Training Loss:  0.349  - Validation loss:  0.330
  20. Epoch  16/100 Batch   50/77 - Training Loss:  0.303  - Validation loss:  0.282
  21. Epoch  17/100 Batch   50/77 - Training Loss:  0.261  - Validation loss:  0.240
  22. Epoch  18/100 Batch   50/77 - Training Loss:  0.226  - Validation loss:  0.203




预测：

   
   
  1. from data_utils import  extract_character_vocab,read_file
  2. import tensorflow as tf
  3. batch_size=128
  4. source_path="data/letters_source.txt"
  5. target_path="data/letters_target.txt"
  6. source_data, target_data=read_file(source_path,target_path)
  7. target_int_to_letter, target_letter_to_int = extract_character_vocab(target_data)
  8. source_int_to_letter, source_letter_to_int = extract_character_vocab(source_data)
  9. def source_to_seq(text):
  10.     '''
  11.     对源数据进行转换
  12.     '''
  13.     sequence_length = 7
  14.     return [source_letter_to_int.get(word, source_letter_to_int['<UNK>']) for word in text] + [source_letter_to_int['<PAD>']]*(sequence_length-len(text))
  17. input_word = 'xxegcbj'
  18. text = source_to_seq(input_word)
  20. checkpoint = "modelsave/trained_model.ckpt"
  22. loaded_graph = tf.Graph()
  23. with tf.Session(graph=loaded_graph) as sess:
  24.     # 加载模型
  25.     loader = tf.train.import_meta_graph(checkpoint + '.meta')
  26.     loader.restore(sess, checkpoint)
  28.     input_data = loaded_graph.get_tensor_by_name('inputs:0')
  29.     logits = loaded_graph.get_tensor_by_name('predictions:0')
  30.     source_sequence_length = loaded_graph.get_tensor_by_name('source_sequence_length:0')
  31.     target_sequence_length = loaded_graph.get_tensor_by_name('target_sequence_length:0')
  33.     answer_logits = sess.run(logits, {input_data: [text] * batch_size,
  34.                                       target_sequence_length: [len(input_word)] * batch_size,
  35.                                       source_sequence_length: [len(input_word)] * batch_size})[0]
  38. print("answer_logits is :",answer_logits)
  39. pad = source_letter_to_int["<PAD>"]
  41. print('原始输入:', input_word)
  43. print('\nSource')
  44. print('  Word 编号:    {}'.format([i for i in text]))
  45. print('  Input Words: {}'.format(" ".join([source_int_to_letter[i] for i in text])))
  47. print('\nTarget')
  48. print('  Word 编号:       {}'.format([i for i in answer_logits if i != pad]))
  49. print('  Response Words: {}'.format(" ".join([target_int_to_letter[i] for i in answer_logits if i != pad])))


旭旭_哥
关注
专栏目录
Seq2Seq in Keras 示例
NeutronT的博客
12-04 1991
示例代码源自keras/examples/lstm_seq2seq.py算法介绍 算法目的是将来自某个域中的输入序列XX（英语语句） 映射到 另一个域中对应的目标序列 YY（法语语句）。 LSTM编码器：将输入序列映射 XX 为一个2维的状态向量vv（仅保留LSTM最后的状态向量，而不要LSTM的输出）； LSTM解码器：训练该解码器，将目标序列 YY 映射到向后偏移一个时间步的 YY 。这个训练过
Seq2Seq 字符集模型的简单实现
10-24
Seq2Seq 包含encoder和decoder模型的简单实现 包括预料库。代码是python3.6+tensorflow1.10，可以直接运行。
OpenSeq2Seq 开源项目教程
最新发布
gitblog_00584的博客
08-22 842
OpenSeq2Seq 开源项目教程 OpenSeq2SeqToolkit for efficient experimentation with Speech Recognition, Text2Speech and NLP项目地址:https://gitcode.com/gh_mirrors/op/OpenSeq2Seq 项目介绍 OpenSeq2Seq 是一个由 NVIDIA 开发的开源工具...
seq2seq模型实现
u014475479的博客
09-05 1405
致谢 找了很久才找到的一个非常不错的seq2seq实现的教程 源代码基于tensorflow 1.6 原文链接 正文 本文是基于TensorFlow 1.6版本的Seq2Seq模型实现了一个机器翻译（Machine Translation）模型的baseline。 本篇代码与去年我在知乎专栏上发表的从Encoder到Decoder实现Seq2Seq模型大同小异，更新的原...
Dynamic seq2seq in tensorflow
weixin_33916256的博客
01-25 94
v1.0中 tensorflow渐渐废弃了老的非dynamic的seq2seq接口，已经放到 tf.contrib.legacy_seq2seq目录下面。 tf.contrib.seq2seq下面的实现都是dynamic seq2seq接口。 按照google的rd说法下个月将会给出更加完善的接口和实现。 当前版本也可以使用这里尝试分析一下现有dynamic seq2seq的代码。   ...
Numpy-based-Model:仅使用numpy的LSTM的Seq2seq模型
05-16
基于数字的模型 仅使用numpy的LSTM的Seq2seq模型
基于python的知识库的问答seq2seq模型代码实现
03-25
综上所述，"基于python的知识库的问答seq2seq模型代码实现"项目涵盖了从数据预处理到模型训练、评估和部署的全过程，旨在创建一个能够从知识库中检索信息并生成准确回答的智能问答系统。通过阅读和理解提供的代码，...
基于seq2seq模型的简单对话系统的tf实现
02-08
**基于Seq2Seq模型的简单对话系统在TensorFlow中的实现** Seq2Seq（Sequence to Sequence）模型是一种在自然语言处理领域广泛应用的深度学习模型，尤其在机器翻译、对话系统等任务上表现出色。该模型由两个关键部分...
基于Python seq2seq模型的简单对话系统的tf实现【100012378】
05-24
**基于Python seq2seq模型的简单对话系统** 在人工智能领域，对话系统是研究的重要方向，它使得机器能够理解和回应人类的自然语言对话。本项目利用TensorFlow（简称tf）框架实现了一个基于seq2seq（sequence to ...
seq2seq模型和基于注意力机制的seq2seq模型
03-21
在实现MNIST数据集的分类任务时，seq2seq模型首先将每个28x28像素的手写数字图像转化为一维的像素序列，然后输入到编码器中。编码器通过RNN（如LSTM或GRU）对序列进行处理，生成一个固定长度的向量。解码器同样采用...
dynamic-seq2seq:seq2seq中文聊天机器人
05-19
dynamic-seq2seq 欢迎关注我的另一个项目 基于中文语料和dynamic_rnn的seq2seq模型 Update: 修复loss计算bug 修复batch_size大于1时的计算bug Requirements tensorflow-1.4+ python2.7 (暂时未对python3 兼容) requests jieba cPickle numpy 谷歌最近开源了一个seq2seq项目 tensorflow推出了dynamic_rnn替代了原来的bucket，本项目就是基于dynamic_rnn的seq2seq模型。 这里我构建了一些对话预料，中文语料本身就比较稀缺，理论上来说语料越多模型的效果越好，但会遇到很多新的问题，这里就不多作说明。 对话语料分别在data目录下 Q.txt A.txt中，可以替换成你自己的对话语料。 用法: # 新增小黄鸡语料 # 添加 pyt
OpenSeq2Seq：使用语音识别，Text2Speech和NLP进行有效实验的工具包
02-05
OpenSeq2Seq：用于序列到序列模型的分布式和混合精度训练的工具包 OpenSeq2Seq的主要目标是使研究人员能够最有效地探索各种序列到序列模型。 通过完全支持分布式和混合精度培训来实现效率。 OpenSeq2Seq使用TensorFlow构建，并为训练用于神经机器翻译，自动语音识别，语音合成和语言建模的编码器-解码器模型提供了所有必要的构建块。 文档和安装说明 产品特点 型号： 神经机器翻译 自动语音识别 语音合成 语言建模 NLP任务（情感分析） 数据并行分布式培训 多GPU 多节点 NVIDIA Volta / Turing GPU的混合精度培训 软件需求 Python>
Seq2Seq数字代码解读
dimaoe2852的博客
08-21 324
# -*- coding: utf-8 -*- import tensorflow as tf import numpy as np import helper ####################################################### 定义模型 PAD = 0 EOS = 1 # UNK = 2 # GO = 3 v...
91.使用注意力机制的seq2seq以及代码实现
weixin_47505105的博客
01-25 1247
笔记
用encoder和decoder实现seq2seq过程
weixin_45169380的博客
02-11 1111
encoder与decoder的介绍 第一个组件是一个编码器（encoder）： 它接受一个长度可变的序列作为输入， 并将其转换为具有固定形状的编码状态。 第二个组件是解码器（decoder）： 它将固定形状的编码状态映射到长度可变的序列。 这被称为编码器-解码器（encoder-decoder）架构 给定一个英文的输入序列：“They”、“are”、“watching”、“.”。 首先，这种“编码器－解码器”架构将长度可变的输入序列编码成一个“状态”， 然后对该状态进行解码， 一个词元接着一个词元
自然语言处理（十四）：从零开始构建使用注意力机制的Seq2Seq网络实现翻译任务
GeniusAng的博客
01-10 1447
自然语言处理（十四）：从零开始构建使用注意力机制的Seq2Seq网络实现翻译任务
Seq2Seq Attention模型
weixin_43758551的博客
07-14 464
https://blog.csdn.net/qq_32241189/article/details/81591456 Seq2Seq模型是RNN最重要的一个变种：N vs M（输入与输出序列长度不同）。 这种结构又叫Encoder-Decoder模型。 原始的N vs N RNN要求序列等长，然而我们遇到的大部分问题序列都是不等长的，如机器翻译中，源语言和目标语言的句子往往并没有相同的长度。 为此...
Seq2Seq模型及代码 好的文章
weixin_44317740的博客
06-05 274
https://mp.weixin.qq.com/s/0k71fKKv2SRLv9M6BjDo4w https://blog.csdn.net/qq_32241189/article/details/81591456?utm_medium=distribute.pc_relevant.none-task-blog-BlogCommendFromMachineLearnPai2-4.nonecase&depth_1-utm_source=distribute.pc_relevant.none-task
Seq2Seq模型代码实现
05-24
下面是一个简单的Seq2Seq模型代码实现，供参考： 首先，导入必要的库： ```python import tensorflow as tf from tensorflow.keras.layers import Input, LSTM, Dense from tensorflow.keras.models import Model ...
评论 2
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值