12.RNN recurrent NEural Network

递归神经网络，循环神经网络，在语音识别，自然语言处理，机器翻译以及图像描述。

如果需要处理文字，语音的问题，需要把他们看成一个连续的整体，所以增加一个反馈回路，把上一个时间的输出信息，作为下一个时间输入作为处理。

本质还是BP神经网络，区别是BP没有反馈神经网络。

深度学习之RNN(循环神经网络)) 讲的非常的不错，可以进行参考

---

LSTM Long short Term Memory

import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_data

mnist = input_data.read_data_sets('MNIST_data', one_hot=True)
#输入图片是28*28
n_inputs = 28  #输入一行，一行有28个像素点
max_time = 28 #一共28行，28列，一次输入1行28个像素点，一共输入28次（28列）
lstm_size = 100  #隐藏单元个数，100个block
n_classes =  10 # 10个分类， 0-9的分类
batch_size = 50 #每批次50个样本
n_batch = mnist.train.num_examples // batch_size

x = tf.placeholder(tf.float32, [None, 784])
y = tf.placeholder(tf.float32, [None, 10])

weights = tf.Variable(tf.truncated_normal([lstm_size, n_classes],  stddev=0.1)) # 行是100 中间隐藏层100*10
biases  = tf.Variable(tf.constant(0.1, shape=[n_classes])) # 10个b

#定义RNN网络
def RNN(X, weights, biases): #X就是 批次的数据  开始是 [50， 784]
    # input=[batch_size,max_time,n_inputs]
    inputs = tf.reshape(X,[-1, max_time, n_inputs])  #reshape之后  [50,28,28]
    # 定义LSTM基本CELL
    # lstm_cell=tf.contrib.rnn.core_rnn_cell.BasicLSTMCell(lstm_size)#版本问题
    lstm_cell  = tf.nn.rnn_cell.BasicLSTMCell(lstm_size) # lstm_size = 100  定义了100个block cell单元
    # final_state[0]是cell state  ？？？
    # final_state[1]是hidden_state ？？？  肯定是整个网络的输出
    outpus, final_state = tf.nn.dynamic_rnn(lstm_cell, inputs, dtype=tf.float32) # inputs格式必须是[-1, max_time, n_inputs]
    results = tf.nn.softmax(tf.matmul(final_state[1], weights) + biases)  #转化为概率
    return  results

#计算RNN的返回结果
prediction = RNN(x, weights, biases)

cross_entropy = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits_v2(labels=y, logits=prediction))
train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy) # 优化cross_entropy
corrent_prediction = tf.equal(tf.argmax(y,1), tf.argmax(prediction, 1))
accuracy = tf.reduce_mean(tf.cast(corrent_prediction, tf.float32))

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    for epoch in range(6):
        for batch in range(n_batch):
            batch_xs, batch_ys = mnist.train.next_batch(batch_size)
            sess.run(train_step, feed_dict={x:batch_xs, y:batch_ys})
        acc = sess.run(accuracy, feed_dict={x:mnist.test.images, y:mnist.test.labels})
        print("Iter " + str(epoch) + ", Testing Accuracy=" + str(acc))

RNN 主要用在文本和语音上面。但是也可以处理分类问题。

Iter 0, Testing Accuracy=0.7175
Iter 1, Testing Accuracy=0.801
Iter 2, Testing Accuracy=0.8752
Iter 3, Testing Accuracy=0.9072
Iter 4, Testing Accuracy=0.9171
Iter 5, Testing Accuracy=0.9238