128篇论文，21大领域，深度学习最值得看的资源全在这里

本文来源：AI科技大本营（rgznai100）
参考 - https://zhuanlan.zhihu.com/p/23080129

从全局到枝干、从经典到前沿、从理论到应用、还有最新的研究…，所有你不需要的需要的，现在不需要的未来需要的，你不需要的周边小伙伴需要的…反正全都在这了。

对于大多数想上手深度学习的小伙伴来说，“我应当从那篇论文开始读起？”

这是一个亘古不变的话题。

而对那些已经入门的同学来说，了解一下不同方向的论文，也是不时之需。

有没有一份完整的深度学习论文导引，让所有人都可以在里面找到想要的内容呢？

有！

今天就给大家分享一篇史上最牛的深度学习论文整合合集。它让大家对整个深度学习领域及其个枝干都能有一个相对完整的理解。

这份阅读列表的组织原则是这样的：

• 从全局到枝干：从综述类、全局性的文章到细分领域的具体论文。
• 从经典到最前沿：每个话题的文章都是按时间顺序来排的，可以清晰给出每个方向的发展脉络。
• 从通用理论到具体应用：有些论文是针对深度学习通用理论的，有些论文章则针对具体的应用领域。
• 专注于最先进的研究：收集有许多最新论文，保证阅读列表的时效性。

当然，这里的每个话题都只选几篇最具代表性的论文，深入研究的话，还需要更进一步的阅读。

基于这些论文的影响力，你会发现很多新近发表的文章也值得一读。此外，这份阅读列表在原文页面会不断更新，值得你时时备查。
https://github.com/songrotek/Deep-Learning-Papers-Reading-Roadmap

想一键打包下载所有的论文？没问题，AI科技大本营已经给你准备好了懒人专属通道。请在公众号会话回复“路径”，即可获取本文所有论文PDF资料。

深度学习基础及历史

1.0 书

[0] 深度学习圣经 ★★★★★
Bengio, Yoshua, Ian J. Goodfellow, and Aaron Courville. “Deep learning.” An MIT Press book. (2015).
https://github.com/HFTrader/DeepLearningBook/raw/master/DeepLearningBook.pdf

1.1 报告

[1] 三巨头报告★★★★★
LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton. “Deep learning.” Nature 521.7553 (2015): 436-444.
http://www.cs.toronto.edu/%7Ehinton/absps/NatureDeepReview.pdf

1.2 深度信念网络 (DBN)

[2] 深度学习前夜的里程碑 ★★★
Hinton, Geoffrey E., Simon Osindero, and Yee-Whye Teh. “A fast learning algorithm for deep belief nets.” Neural computation 18.7 (2006): 1527-1554.
http://www.cs.toronto.edu/%7Ehinton/absps/ncfast.pdf

[3] 展示深度学习前景的里程碑 ★★★
Hinton, Geoffrey E., and Ruslan R. Salakhutdinov. “Reducing the dimensionality of data with neural networks.” Science 313.5786 (2006): 504-507.
http://www.cs.toronto.edu/%7Ehinton/science.pdf

1.3 ImageNet革命（深度学习大爆炸）

[4] AlexNet的深度学习突破 ★★★
Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. “Imagenet classification with deep convolutional neural networks.” Advances in neural information processing systems. 2012.
http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf

[5] VGGNet深度神经网络出现 ★★★
Simonyan, Karen, and Andrew Zisserman. “Very deep convolutional networks for large-scale image recognition.” arXiv preprint arXiv:1409.1556 (2014).
https://arxiv.org/pdf/1409.1556.pdf

[6] GoogLeNet ★★★
Szegedy, Christian, et al. “Going deeper with convolutions.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015.
http://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Szegedy_Going_Deeper_With_2015_CVPR_paper.pdf

[7] ResNet极深度神经网络，CVPR最佳论文 ★★★★★
He, Kaiming, et al. “Deep residual learning for image recognition.” arXiv preprint arXiv:1512.03385 (2015).
https://arxiv.org/pdf/1512.03385.pdf

1.4 语音识别革命

[8] 语音识别突破 ★★★★
Hinton, Geoffrey, et al. “Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups.” IEEE Signal Processing Magazine 29.6 (2012): 82-97.
http://cs224d.stanford.edu/papers/maas_paper.pdf

[9] RNN论文 ★★★
Graves, Alex, Abdel-rahman Mohamed, and Geoffrey Hinton. “Speech recognition with deep recurrent neural networks.” 2013 IEEE international conference on acoustics, speech and signal processing. IEEE, 2013.
http://arxiv.org/pdf/1303.5778.pdf

[10] 端对端RNN语音识别 ★★★
Graves, Alex, and Navdeep Jaitly. “Towards End-To-End Speech Recognition with Recurrent Neural Networks.” ICML. Vol. 14. 2014.
http://www.jmlr.org/proceedings/papers/v32/graves14.pdf

[11] Google语音识别系统论文 ★★★
Sak, Haşim, et al. “Fast and accurate recurrent neural network acoustic models for speech recognition.” arXiv preprint arXiv:1507.06947 (2015).
http://arxiv.org/pdf/1507.06947

[12] 百度语音识别系统论文 ★★★★
Amodei, Dario, et al. “Deep speech 2: End-to-end speech recognition in english and mandarin.” arXiv preprint arXiv:1512.02595 (2015).
https://arxiv.org/pdf/1512.02595.pdf

[13] 来自微软的当下最先进的语音识别论文 ★★★★
W. Xiong, J. Droppo, X. Huang, F. Seide, M. Seltzer, A. Stolcke, D. Yu, G. Zweig “Achieving Human Parity in Conversational Speech Recognition.” arXiv preprint arXiv:1610.05256 (2016).
https://arxiv.org/pdf/1610.05256v1

读完上面这些论文，你将对深度学习的历史、深度学习模型（CNN、RNN、LSTM等）的基本架构有一个基本认识，并能理解深度学习是如何解决图像及语音识别问题的。接下来的论文将带你深入理解深度学习方法、深度学习在前沿领域的不同应用。根据自己的兴趣和研究方向选择阅读即可：

深度学习方法

2.1 模型

[14] Dropout ★★★
Hinton, Geoffrey E., et al. “Improving neural networks by preventing co-adaptation of feature detectors.” arXiv preprint arXiv:1207.0580 (2012).
https://arxiv.org/pdf/1207.0580.pdf

[15] 过拟合 ★★★
Srivastava, Nitish, et al. “Dropout: a simple way to prevent neural networks from overfitting.” Journal of Machine Learning Research 15.1 (2014): 1929-1958.
http://www.jmlr.org/papers/volume15/srivastava14a.old/source/srivastava14a.pdf

[16] Batch归一化——2015年杰出成果 ★★★★
Ioffe, Sergey, and Christian Szegedy. “Batch normalization: Accelerating deep network training by reducing internal covariate shift.” arXiv preprint arXiv:1502.03167 (2015).
http://arxiv.org/pdf/1502.03167

[17] Batch归一化的升级 ★★★★
Ba, Jimmy Lei, Jamie Ryan Kiros, and Geoffrey E. Hinton. “Layer normalization.” arXiv preprint arXiv:1607.06450 (2016).
https://arxiv.org/pdf/1607.06450.pdf

[18] 快速训练新模型 ★★★
Courbariaux, Matthieu, et al. “Binarized Neural Networks: Training Neural Networks with Weights and Activations Constrained to+ 1 or−1.”
https://pdfs.semanticscholar.org/f832/b16cb367802609d91d400085eb87d630212a.pdf

[19] 训练方法创新 ★★★★★
Jaderberg, Max, et al. “Decoupled neural interfaces using synthetic gradients.” arXiv preprint arXiv:1608.05343 (2016).
https://arxiv.org/pdf/1608.05343

[20] 修改预训练网络以降低训练耗时 ★★★
Chen, Tianqi, Ian Goodfellow, and Jonathon Shlens. “Net2net: Accele