基于seq2seq的时间序列预测实验（转）

基于seq2seq的时间序列预测实验（转）

https://zhuanlan.zhihu.com/p/39140472

本文使用seq2seq模型来做若干组时间序列的预测任务，目的是验证RNN这种网络结构对时间序列数据的pattern的发现能力，并在小范围内探究哪些pattern是可以被识别的，哪些pattern是无法识别的。

本文是受github上一个项目的启发，这个项目是做时间序列信号的预测。我借用了一部分他的代码，做的实验与他不同，是自己的想法，放上来和大家交流讨论。

guillaume-chevalier/seq2seq-signal-prediction

---

下面将进行若干组的实验。方法是每组实验生成一系列的随机时间序列数据作为训练数据，时间序列数据的长度是seq_length * 2， 我们用seq2seq模型的encoder学习出前半段（长度为seq_length）序列的pattern，然后用decoder的输出作为对后半段数据的预测，我们希望decoder的输出越接近后半段的真实数据越好。

实验一：平稳时间序列

如下图，前半段数据与后半段数据的长度相等，而且后半段的高度比前半段总是高出1。看过大量的这种类型的数据后，你能识别出“后半段的高度比前半段总是高出1”这种模式进而准确预测出后半段的数据吗？看起来这是很简单的模式识别问题，之所以简单，是因为前半段和后半段的数据都是常量，只是后半段比前半段大一个常量罢了。

为了做实验，我们需要生成大量的这种数据，并搭建seq2seq模型(使用tensorflow)，下面贴出主要的代码。

生成总长度为30，即前半段、后半段长度均为15，后半段高度总比前半段高度大1的随机时间序列数据，

def generate_x_y_data_two_freqs(batch_size, seq_length):
    batch_x = []
    batch_y = [] for _ in range(batch_size): amp_rand = random.random() a = np.asarray([amp_rand] * seq_length) b = np.asarray([amp_rand + 1] * seq_length) sig = np.r_[a,b] x1 = sig[:seq_length] y1 = sig[seq_length:] x_ = np.array([x1]) y_ = np.array([y1]) x_, y_ = x_.T, y_.T batch_x.append(x_) batch_y.append(y_) batch_x = np.array(batch_x) batch_y = np.array(batch_y) batch_x = np.array(batch_x).transpose((1, 0, 2)) batch_y = np.array(batch_y).transpose((1, 0, 2)) return batch_x, batch_y def generate_x_y_data_v2(batch_size): return generate_x_y_data_two_freqs(batch_size, seq_length=15) 

用tensorflow搭建seq2seq，其中encoder和decoder使用相同结构(但不share weights)的双层GRU。这块的代码是直接使用上文提到的github项目代码，

tf.reset_default_graph()
# sess.close()
sess = tf.InteractiveSession()

layers_stacked_count = 2

with tf.variable_scope('Seq2seq'): # Encoder: inputs enc_inp = [ tf.placeholder(tf.float32, shape=(None, input_dim), name="inp_{}".format(t)) for t in range(seq_length) ] # Decoder: expected outputs expected_sparse_output = [ tf.placeholder(tf.float32, shape=(None, output_dim), name="expected_sparse_output_".format(t)) for t in range(seq_length) ] # Give a "GO" token to the decoder. # Note: we might want to fill the encoder with zeros or its own feedback rather than with "+ enc_inp[:-1]" dec_inp = [ tf.zeros_like(enc_inp[0], dtype=np.float32, name="GO") ] + enc_inp[:-1] # Create a `layers_stacked_count` of stacked RNNs (GRU cells here). cells = [] for i in range(layers_stacked_count): with tf.variable_scope('RNN_{}'.format(i)): cells.append(tf.nn.rnn_cell.GRUCell(hidden_dim)) # cells.append(tf.nn.rnn_cell.BasicLSTMCell(...)) cell = tf.nn.rnn_cell.MultiRNNCell(cells) # Here, the encoder and the decoder uses the same cell, HOWEVER, # the weights aren't shared among the encoder and decoder, we have two # sets of weights created under the hood according to that function's def. dec_outputs, dec_memory = tf.nn.seq2seq.basic_rnn_seq2seq( enc_inp, dec_inp, cell ) # For reshaping the output dimensions of the seq2seq RNN: w_out = tf.Variable(tf.random_normal([hidden_dim, output_dim])) b_out = tf.Variable(tf.random_normal([output_dim])) # Final outputs: with linear rescaling for enabling possibly large and unrestricted output values. output_scale_factor = tf.Variable(1.0, name="Output_ScaleFactor") reshaped_outputs = [output_scale_factor*(tf.matmul(i, w_out) + b_out) for i in dec_outputs] # Training loss and optimizer with tf.variable_scope('Loss'): # L2 loss output_loss = 0 for _y, _Y in zip(reshaped_outputs, expected_sparse_output): output_loss += tf.reduce_mean(tf.nn.l2_loss(_y - _Y)) # L2 regularization (to avoid overfitting and to have a better generalization capacity) reg_loss = 0 for tf_var in tf.trainable_variables(): if not ("Bias" in tf_var.name