一直在等待，一直会等待 RNN系列--3

1、tf.nn.dynamic_rnn

tf.nn.dynamic_rnn(
    cell,
    inputs,
    sequence_length=None,
    initial_state=None,
    dtype=None,
    parallel_iterations=None,
    swap_memory=False,
    time_major=False,
    scope=None
)

         功能： 通过给定的RNNCell创建递归神经网络，实现输入的全自动展开
         参数：
cell: RNNCell实例
inputs: RNNCell的输入
initial_state: RNN的初始状态，如果cell.state_size为整数，形状为[batch_size, cell.state_size]. 如果cell.state_size为元组tuple，形状为[batch_size, s] for s in cell.state_size.
time_major: 输入输出形状格式，为True时，[max_time, batch_size, depth]；为False时，[batch_size, max_time, depth]

         返回值： (outputs, state)
outputs： 输出值，输出格式由time_major决定
state： 最后隐层的状态值
         cell.state_size为整数，shape为 [batch_size, cell.state_size]；否则shaped为[batch_size] + cell.state_size.

         注意： RNNCell的两个重要类属性
state_size： 隐层的大小
output_size： 输出的大小
Keras关于LSTM的units参数： https://www.zhihu.com/question/64470274/answer/256379387

        单层RNN代码示例：

# create a BasicRNNCell，隐层大小设置为hidden_size，此处解释为Keras关于LSTM的units参数
rnn_cell = tf.nn.rnn_cell.BasicRNNCell(hidden_size)

# 'outputs' is a tensor of shape [batch_size, max_time, cell_state_size]
# 'inputs' is a tensor of shape [batch_size, max_time, input_size]
# max_time是序列本身的长度，如果为句子，则为10个单词；input_size是输入数据单个序列单个时间维度上固有的长度；

# defining initial state，即时间为0时，RNNCell的状态
initial_state = rnn_cell.zero_state(batch_size, dtype=tf.float32)

# 'state' is a tensor of shape [batch_size, cell_state_size]
outputs, state = tf.nn.dynamic_rnn(rnn_cell, input_data,
                                   initial_state=initial_state,
                                   dtype=tf.float32)

        多层RNN代码示例：

# create 2 LSTMCells
rnn_layers = [tf.nn.rnn_cell.LSTMCell(size) for size in [128, 256]]

# create a RNN cell composed sequentially of a number of RNNCells
multi_rnn_cell = tf.nn.rnn_cell.MultiRNNCell(rnn_layers)

# 'outputs' is a tensor of shape [batch_size, max_time, 256]
# 'state' is a N-tuple where N is the number of LSTMCells containing a
# tf.contrib.rnn.LSTMStateTuple for each cell
outputs, state = tf.nn.dynamic_rnn(cell=multi_rnn_cell,
                                   inputs=data,
                                   dtype=tf.float32)

2、tf.nn.rnn_cell.DropoutWrapper

__init__(
    cell,
    input_keep_prob=1.0,
    output_keep_prob=1.0,
    state_keep_prob=1.0,
    variational_recurrent=False,
    input_size=None,
    dtype=None,
    seed=None,
    dropout_state_filter_visitor=None
)

tensorflow.python.array_ops.split

@tf_export("split")
def split(value, num_or_size_splits, axis=0, num=None, name="split"):
Splits a tensor into sub tensors.

  If `num_or_size_splits` is an integer type, then `value` is split
  along dimension `axis` into `num_split` smaller tensors.
  Requires that `num_split` evenly divides `value.shape[axis]`.

  If `num_or_size_splits` is not an integer type, it is presumed to be a Tensor
  `size_splits`, then splits `value` into `len(size_splits)` pieces. The shape
  of the `i`-th piece has the same size as the `value` except along dimension
  `axis` where the size is `size_splits[i]`.

  For example:


  # 'value' is a tensor with shape [5, 30]
  # Split 'value' into 3 tensors with sizes [4, 15, 11] along dimension 1
  split0, split1, split2 = tf.split(value, [4, 15, 11], 1)
  tf.shape(split0)  # [5, 4]
  tf.shape(split1)  # [5, 15]
  tf.shape(split2)  # [5, 11]
  
  # Split 'value' into 3 tensors along dimension 1
  split0, split1, split2 = tf.split(value, num_or_size_splits=3, axis=1)
  tf.shape(split0)  # [5, 10]

tf.nn.rnn_cell.LSTMStateTuple

         Tuple used by LSTM Cells for state_size, zero_state, and output state. Stores two elements: (c, h), in that order. Where c is the hidden state and h is the output. Only used when state_is_tuple=True.