【keras】rnn中的LSTM

keras rnn中常见的rnn layer

1. LSTM

LSTM内部结构如下，

公式为
$$\begin{gathered} inputgate:i^t=\sigma (W^i{x}^t+U^i{h}^{(t-1)}) \\ forgetgate:f^t=\sigma (W^f{x}^t+U^f{h}^{(t-1)}) \\ output:o^t=\sigma (W^o{x}^t+U^o{h}^{(t-1)}) \\ Newmemorycell:{\tilde{c}}^t=tanh(W^c{x}^t+U^c{x}^t) \\ Finalmemorycell:c^t=f^t\odot c^{(t-1)}+i^t\odot {\tilde{c}}^t \\ hiddenstate:h^t=o^t\odot tanh(c^t) \end{gathered}$$

它的每一个time step都有三个输入与两个输出，
输入为上一个time step的h(t-1),c(t-1),x(t)
输出为h(t),c(t)

---

下面我们以一个例子来看一下keras lstm的使用

from keras.models import Model
from keras.layers import Input
from keras.layers import LSTM
import numpy as np

#               t1   t2   t3   t4
src = np.array([0.1, 0.2, 0.3, 0.4])

# define model
inputs1 = Input(shape=(4,1))
# 1: Positive integer, dimensionality of the output space
lstm1= LSTM(1)(inputs1) 
model = Model(inputs=inputs1, outputs=lstm1)  
# define input data
data = src.reshape((1,4,1))
print("====================")
print(model.predict(data))

结果如下，

====================
[[0.11041524]]  # h(4)

简单说明一下参数：

• LSTM的参数1，是说明W,U向量的维数。根据上面的公式，一维的参数向量最终计算出来的结果就是一个标量。 它也是最后一个time step的输出h(t)

使用return_sequences和return_state参数

使用return_sequences=True

from keras.models import Model
from keras.layers import Input
from keras.layers import LSTM
from keras.layers import GRU
import numpy as np

#               t1   t2   t3   t4
src = np.array([0.1, 0.2, 0.3, 0.4])

# define model
inputs1 = Input(shape=(4,1))
lstm1= LSTM(1, return_sequences=True)(inputs1)
model = Model(inputs=inputs1, outputs=lstm1) 
data = src.reshape((1,4,1))
print("====================")
print(model.predict(data))

结果如下：

====================
[[[-0.02509235]  # the last h(4)
  [-0.06933878]  # h(3)
  [-0.12798117]  # h(2)
  [-0.19663112]]] # h(1)

它输出的是每个节点的output。下面是官方注释说明。

return_sequences: Boolean. Whether to return the last output
in the output sequence, or the full sequence.

使用return_state=True

from keras.models import Model
from keras.layers import Input
from keras.layers import LSTM
from keras.layers import GRU
import numpy as np

#               t1   t2   t3   t4
src = np.array([0.1, 0.2, 0.3, 0.4])

# define model
inputs1 = Input(shape=(4,1))
lstm1= LSTM(1, return_state=True)(inputs1)
model = Model(inputs=inputs1, outputs=lstm1) 
data = src.reshape((1,4,1))
print("====================")
print(model.predict(data))

结果

====================
[array([[-0.11774988]], dtype=float32),  # the last output h(4)
array([[-0.11774988]], dtype=float32),   # the additional information for the last state. h(4)
array([[-0.24601391]], dtype=float32)]   # the state for h(4): c(4)

return_state: Boolean. Whether to return the last state in addition to the output.