1、循环神经网络的模型
一般进行如下操作:
输入得到;然后输入得到;然后输入得到,以此类推;那么如何通过Tensorflow一步实现呢?
2. dynamic_rnn
为了实现一步计算多次,我们就要用到Tensorflow中的dynamic_rnn(),代码如下(实现了上图列出的三步):
import tensorflow as tf
import numpy as np
from tensorflow.python.ops import variable_scope as vs
output_size = 5
batch_size = 4
time_step = 3
dim = 3
cell = tf.nn.rnn_cell.BasicRNNCell(num_units=output_size)
inputs = tf.placeholder(dtype=tf.float32, shape=[time_step, batch_size, dim])
h0 = cell.zero_state(batch_size=batch_size, dtype=tf.float32)
X = np.array([[[1, 2, 1], [2, 0, 0], [2, 1, 0], [1, 1, 0]], # x1
[[1, 2, 1], [2, 0, 0], [2, 1, 0], [1, 1, 0]], # x2
[[1, 2, 1], [2, 0, 0], [2, 1, 0], [1, 1, 0]]]) # x3
outputs, final_state = tf.nn.dynamic_rnn(cell, inputs, initial_state=h0, time_major=True)
sess = tf.Session()
sess.run(tf.global_variables_initializer())
a, b = sess.run([outputs, final_state], feed_dict={inputs:X})
print(a)
print(b)
其中第7行time_step=3就表示计算三步,所以输入X就对应有三个部分。再最终的输出结果中,outputs里包含了(outputs1,outputs2,outputs3),而final_stat就只是h3,并且outputs3,h3 是相等的。
outputs:
[[[ 0.9427065 -0.92617476 -0.79179853 0.6308035 0.07298201]
[ 0.7051633 -0.62077284 -0.79618317 0.5004738 -0.20110159]
[ 0.85066974 -0.77197933 -0.76875883 0.80251306 -0.04951192]
[ 0.67497337 -0.57974416 -0.4408107 0.68083197 0.05233984]]# output1
[[ 0.9828192 -0.9433205 -0.9233751 0.72930676 -0.34445292]
[ 0.92153275 -0.58029604 -0.8949743 0.5431045 -0.46945637]
[ 0.9690989 -0.7922626 -0.8973758 0.81312704 -0.46288016]
[ 0.88565385 -0.6617377 -0.68075943 0.70066273 -0.34827012]]# output2
[[ 0.99172366 -0.93298715 -0.9272905 0.7158564 -0.46278387]
[ 0.9566409 -0.5595625 -0.9101479 0.58005375 -0.5905321 ]
[ 0.9838727 -0.7693646 -0.91019756 0.82892674 -0.58026373]
[ 0.9438508 -0.61732507 -0.7356022 0.73460865 -0.483655 ]]]# output3
final_state:
[[ 0.99172366 -0.93298715 -0.9272905 0.7158564 -0.46278387]
[ 0.9566409 -0.5595625 -0.9101479 0.58005375 -0.5905321 ]
[ 0.9838727 -0.7693646 -0.91019756 0.82892674 -0.58026373]
[ 0.9438508 -0.61732507 -0.7356022 0.73460865 -0.483655 ]]# final_satae
3.总结
当使用dynamic_rnn时,对于输入数据的格式有两种:
第一种:输入格式为[batch_size,time_steps,input_size],此时得到的输出output的形状为[batch_size,time_steps,output_size],final_state的形状为[batch_size,state_size]
第二种:也就是我们上面用到的,此时的输入格式为[time_steps,batch_size,input_size],得到的输出output的形状为[time_steps,batch_size,output_size],final_state的形状仍然为[batch_size,state_size],但此时要指定time_major = True
对比这两种输入方式,第二种最大的优点就是输出的结果形式方便我们观察。