Note - Sequence models - Recurrent Neural Networks (deeplearning.ai)

课程链接：序列模型 - 网易云课堂 (163.com)

1.1 Why sequence models?

应用：情感分类、机器翻译、命名实体识别……

1.2 Notation

representing words

vocabulary (most common words)

one-hot representation

1.3 Recurrent Neural Network Model

Uni-directional RNN: an limitation is that it uses information earlier in the sentence.

activation functions: tanh/ReLU

1.4 Backpropagation through time

1.5 Different types of RNNs

many-to-many: name entity recognition (input length = output length), machine translation (input length != output length)

many-to-one: sentiment classification

one-to-one

one-to-many: music generation

1.6 Language model and sequence generation

1.7 Sampling novel sequences

1.8 Vanishing gradients with RNNs

Basic RNN not able to capture long-term dependencies

exploding gradients (easier to spot)

1.9 Gated Recurrent Unit (GRU) 门控循环单元

修改RNN隐层，改善梯度消失问题

C = memory cell  更新与否

1.10 LSTM (long short term memory unit) 长短时记忆网络

1.11 Bidirectional RNN (BRNN)

1.12 Deep RNNs