Keras之LSTM源码阅读笔记

这里目前为止只是博主阅读Keras中LSTM源码的草稿笔记，内容不全，没有清晰的逻辑，只是堆砌个人想法。

---

参考文献：
1. keras的官方相关文档
2. LSTM原论文
3. keras的RNN源码

1. 接口研究

1.1. Recurrent接口

Recurrent是LSTM的父类（实际是通过SimpleRNN间接继承），定义所有RNNs的统一接口。

1.1.1. implementation:

implementation: one of {0, 1, or 2}.
If set to 0, the RNN will use an implementation that uses fewer, larger matrix products, thus running faster on CPU but consuming more memory. If set to 1, the RNN will use more matrix products, but smaller ones, thus running slower (may actually be faster on GPU) while consuming less memory. If set to 2 (LSTM/GRU only), the RNN will combine the input gate, the forget gate and the output gate into a single matrix, enabling more time-efficient parallelization on the GPU. Note: RNN dropout must be shared for all gates, resulting in a slightly reduced regularization.
博主主要使用GPU加速且不在意内存的代码，所以通常设置implementation=2，源码阅读也主要集中在implementation=2的部分。

1.1.2. weights:

weights: list of Numpy arrays to set as initial weights.
The list should have 3 elements, of shapes: [(input_dim, output_dim), (output_dim, output_dim), (output_dim,)]`.

1.2. LSTM接口

1.2.1. recurrent_activation

Activation function to use for the recurrent step.
注意： 默认值是’hard_sigmoid’，而原论文中用的’sigmoid’（关于hard_sigmoid 和sigmiod的比较请参考