1. global attention
1. local attention
In concrete details, the model first generates an aligned position p_t for each target word at time t. The context vector ct is then derived as a weighted average over the set of source hidden states within the window [p_t−D, p_t+D]; D is empirically selected.
参考文献:
1. 论文 | Effective Approaches to Attention-based Neural Machine Translation