课程链接:序列模型 - 网易云课堂 (163.com)
1.1 Why sequence models?
应用:情感分类、机器翻译、命名实体识别……
1.2 Notation
representing words
vocabulary (most common words)
one-hot representation
1.3 Recurrent Neural Network Model
Uni-directional RNN: an limitation is that it uses information earlier in the sentence.
activation functions: tanh/ReLU
1.4 Backpropagation through time
1.5 Different types of RNNs
many-to-many: name entity recognition (input length = output length), machine translation (input length != output length)
many-to-one: sentiment classification
one-to-one
one-to-many: music generation
1.6 Language model and sequence generation
1.7 Sampling novel sequences
1.8 Vanishing gradients with RNNs
Basic RNN not able to capture long-term dependencies
exploding gradients (easier to spot)
1.9 Gated Recurrent Unit (GRU) 门控循环单元
修改RNN隐层,改善梯度消失问题
C = memory cell 更新与否
1.10 LSTM (long short term memory unit) 长短时记忆网络
1.11 Bidirectional RNN (BRNN)
1.12 Deep RNNs