1. Transformer
Sequence-to-Sequenct (Seq2seq): input a sequence,output a sequence,输出长度由机器决定。
e.g 语音识别,机器翻译,语音翻译
1. Encoder
1.1. 网络架构
每个block更为具体的架构:
1.2. to learn more
2. Decoder
2.1. 网络架构: Autoregressive
Masked Self-attention:
2.2. AT v.s. NAT
2.3. Cross-attention
2.4. Train
用ground truth做input来train
2.5. Train Tips
- Copy Mechanism
- Guided Attention
- Beam Search
- Samlping