讨论:
https://www.jiqizhixin.com/articles/2017-06-28-5
https://ricardokleinklein.github.io/2017/11/16/Attention-is-all-you-need.html
1. Mutli GPU 和 Single 配置的区别
https://github.com/tensorflow/tensor2tensor/issues/124
https://github.com/tensorflow/tensor2tensor/issues/17
2. 用Multi GUP在跑相同step的速度比Single GPU慢
https://github.com/tensorflow/tensor2tensor/issues/146
https://github.com/tensorflow/tensor2tensor/issues/390
3. batch size参数
https://github.com/tensorflow/tensor2tensor/issues/17#issuecomment-310268149