Transformer技术路线综述——Paddle

本篇为《深度学习》系列博客的第十三篇，该系列博客主要记录深度学习相关知识的学习过程和自己的理解，方便以后查阅。

本篇ppt来自于百度的《百度顶会论文复现营第2期》课程，如想了解详情，请移步上述课程链接。

传统注意力机制

Transformer注意力机制

Transformer在CV中的应用

Wu B , Xu C , Dai X , et al. Visual Transformers: Token-based Image Representation and Processing for Computer Vision[J]. 2020. https://arxiv.org/abs/2006.03677

Dosovitskiy, A. , et al. “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale.” (2020). https://arxiv.org/abs/2010.11929

Han, K. , et al. “Transformer in Transformer.” (2021). http://arxiv.org/abs/2103.00112
Zheng, S. , et al. “Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers.” (2020). https://arxiv.org/abs/2012.15840

Carion, N. , et al. “End-to-End Object Detection with Transformers.” (2020). https://arxiv.org/abs/2005.12872

Zou, Cheng , et al. “End-to-End Human Object Interaction Detection with HOI Transformer.” (2021). https://arxiv.org/abs/2103.04503

Zhu, X. , et al. “Deformable DETR: Deformable Transformers for End-to-End Object Detection.” (2020). http://arxiv.org/abs/2010.04159v4

Chen, H. , et al. “Pre-Trained Image Processing Transformer.” (2020). https://arxiv.org/abs/2012.00364

Han, K. , et al. “A Survey on Visual Transformer.” (2020). http://arxiv.org/abs/2012.12556

Transformer实践