文献 | 2010-2016年被引用次数最多的深度学习论文（修订版）

一、书籍

Deep learning (2015)

作者：Bengio

下载地址：http://www.deeplearningbook.org/

---

二、理论

1.在神经网络中提取知识

Distilling the knowledge in a neural network

作者：G. Hinton et al.

2.深度神经网络很易受骗：高信度预测无法识别的图片

Deep neural networks are easily fooled: High confidence predictions for unrecognizable images

作者：A. Nguyen et al.

3.深度神经网络特征的可迁移性如何？

How transferable are features in deep neural networks? (2014),

作者：J. Yosinski et al.

4.深挖卷积网络的各个细节

Return of the Devil in the Details: Delving Deep into Convolutional Nets (2014)

作者：K. Chatfield et al.

5.为什么无监督预训练对深度学习有帮助？

Why does unsupervised pre-training help deep learning (2010)

作者：D. Erhan et al. (Bengio)

6.理解训练深度前馈神经网络的难点

Understanding the difficulty of training deep feedforward neural networks (2010)

作者：X. Glorot and Y. Bengio

---

三、优化/网络结构

　　简介：本部分从文献7到文献14为神经网络优化的一些方法，尤其是文献7的批归一化更是在业界产生巨大的影响；文献15到文献22为网络结构的变化，包括全卷积神经网络等。这些参考文献都是非常具有参考价值的干货！

7.Batch Normalization 算法：通过减少内部协变量转化加速深度网络的训练（推荐）

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift (2015)

作者：S. Loffe and C. Szegedy (Google)

8.Dropout：一个预防神经网络过拟合的简单方式

Dropout: A simple way to prevent neural networks from overfitting (2014)

作者：N. Srivastava et al. (Hinton)

9.Adam：一个随机优化的方法

Adam: A method for stochastic optimization (2014)

作者：D. Kingma and J. Ba

10.论深度学习领域初始化和动量的重要性

On the importance of initialization and momentum in deep learning (2013)

作者：I. Sutskever et al. (Hinton)

11.使用 Dropconnect 的神经网络正则化

Regularization of neural networks using dropconnect (2013)

作者：L. Wan et al. (LeCun)

12.超参数最优化的随机搜索

Random search for hyper-parameter optimization (2012)

作者：J. Bergstra and Y. Bengio

13.图像识别中的深度残差学习

Deep residual learning for image recognition (2016)

作者：K. He et al. (Microsoft)

14.用于物体精准检测和分割的基于区域的卷积网络

Region-based convolutional networks for accurate object detection and segmentation (2016)

作者：R. Girshick et al.(Microsoft)

15.更深的卷积网络

Going deeper with convolutions (2015)

作者：C. Szegedy et al. (Google)

16.快速 R-CNN 网络

Fast R-CNN (2015)

作者： R. Girshick (Microsoft)

16.更快速的 R-CNN 网络：使用区域网络的实时物体检测

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks (2015)

作者： S. Ren et al.

17.用于语义分割的全卷积神经网络

Fully convolutional networks for semantic segmentation (2015)

作者：J. Long et al.

18.大规模图像识别的深度卷积网络

Very deep convolutional networks for large-scale image recognition (2014)

作者：K. Simonyan and A. Zisserman

19.OverFeat：使用卷积网络融合识别、本地化和检测

OverFeat: Integrated recognition, localization and detection using convolutional networks (2014)

作者：P. Sermanet et al.(LeCun)

20.可视化以及理解卷积网络

Visualizing and understanding convolutional networks (2014)

作者：M. Zeiler and R. Fergus

21.Maxout 网络

Maxout networks (2013)

作者：I. Goodfellow et al. (Bengio)

22.Network In Network 深度网络架构

Network in network (2013)

作者：M. Lin et al.

---

四、图像

１.使用卷积神经网络在自然环境下阅读文本

Reading text in the wild with convolutional neural networks (2016)

作者：M. Jaderberg et al. (DeepMind)

２.Imagenet 大规模视觉识别挑战赛

Imagenet large scale visual recognition challenge (2015)

作者：O. Russakovsky et al.

3.DRAW：一个用于图像生成的循环神经网络

DRAW: A recurrent neural network for image generation (2015)

作者：K. Gregor et al.

4.对精确的物体检测和语义切割更为丰富的特征分层

Rich feature hierarchies for accurate object detection and semantic segmentation (2014)

作者： R. Girshick et al.

5.使用卷积神经网络学习和迁移中层图像表征

Learning and transferring mid-Level image representations using convolutional neural networks (2014)

作者：M. Oquab et al.

6.DeepFace：在面部验证任务中接近人类表现

DeepFace: Closing the Gap to Human-Level Performance in Face Verification (2014)

作者：Y. Taigman et al. (Facebook)

---

五、视频 / 人类行为

1.利用卷积神经网络进行大规模视频分类（2014）

Large-scale video classification with convolutional neural networks (2014)

作者：A. Karpathy et al. (FeiFei)

2.DeepPose：利用深度神经网络评估人类姿势

DeepPose: Human pose estimation via deep neural networks (2014)

作者：A. Toshev and C. Szegedy (Google)

3.用于视频中动作识别的双流卷积网络

Two-stream convolutional networks for action recognition in videos (2014)

作者：K. Simonyan et al.

4.用于人类动作识别的 3D 卷积神经网络（这篇文章针对连续视频帧进行处理，是个不错的）

3D convolutional neural networks for human action recognition (2013)

作者：S. Ji et al.

5.带有改进轨迹的动作识别

Action recognition with improved trajectories (2013)

作者：H. Wang and C. Schmid

6.用独立子空间分析，学习用于动作识别的等级恒定的时空特征

Learning hierarchical invariant spatio-temporal features for action recognition with independent subspace analysis (2011)

作者：Q. Le et al

---

六、自然语言处理

1.用 RNN 编码——解码器学习短语表征，实现统计机器翻译

Learning phrase representations using RNN encoder-decoder for statistical machine translation (2014)

作者：K. Cho et al.(Bengio)

2.一个为句子建模的卷积神经网络

A convolutional neural network for modelling sentences (2014)

作者：N. Kalchbrenner et al.

3.用于句子分类的卷积神经网络

Convolutional neural networks for sentence classification (2014)

作者：Y. Kim

4.斯坦福 coreNLP 自然语言处理工具

The stanford coreNLP natural language processing toolkit (2014)

作者：C. Manning et al.

5.基于情感树库应用于情感组合研究的递归深度网络模型

Recursive deep models for semantic compositionality over a sentiment treebank (2013)

作者：R. Socher et al.

6.基于语言模型的循环神经网络

Recurrent neural network based language model (2010)

作者：T. Mikolov et al.

7.自动语音识别：一种深度学习的方法

Automatic Speech Recognition - A Deep Learning Approach (Book, 2015)

作者：D. Yu and L. Deng (Microsoft)

8.使用深度循环网络进行语音识别

Speech recognition with deep recurrent neural networks (2013)

作者：A. Graves (Hinton)

9.基于上下文预训练的深度神经网络在大规模词表语音识别中的应用

Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition (2012)

作者：G. Dahl et al.

10.使用深度信念网络进行声学建模

Acoustic modeling using deep belief networks (2012)

作者：A. Mohamed et al. (Hinton)

---

七、无监督学习

1.自编码变量贝叶斯

Auto-Encoding Variational Bayes (2013)

作者：D. Kingma and M. Welling

2.用大规模无监督学习搭建高水平特征

Building high-level features using large scale unsupervised learning (2013)

作者：Q. Le et al.

3.无监督特征学习中单层网络分析

An analysis of single-layer networks in unsupervised feature learning (2011)

作者：A. Coates et al.

4.堆栈降噪解码器：在本地降噪标准的深度网络中学习有用的表征

Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion (2010)

作者：P. Vincent et al. (Bengio)

5.训练受限波兹曼机的实践指南

A practical guide to training restricted boltzmann machines (2010)

作者：G. Hinton

---

八、开源架构

1.TensorFlow：异构分布式系统上的大规模机器学习

TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems (2016)

作者：M. Abadi et al. (Google)

2.Theano：一个针对快速计算数学表达公式的Python框架

Theano: A Python framework for fast computation of mathematical expressions

作者：R. Al-Rfou et al. (Bengio)

3.MatConvNet: 针对matlab 的卷积神经网络

MatConvNet: Convolutional neural networks for matlab (2015)

作者：A. Vedaldi and K. Lenc

4.Caffe：快速特征嵌入的卷积结构

Caffe: Convolutional architecture for fast feature embedding (2014)
作者： Y. Jia et al.

---

九、2016最新论文

1.对立学习推论

Adversarially Learned Inference (2016)

作者：V. Dumoulin et al.

2.理解卷积神经网络

Understanding Convolutional Neural Networks (2016)

作者：J. Koushik

3.SqueezeNet 模型：达到 AlexNet 水平的准确率，却使用缩减 50 倍的参数以及< 1MB 的模型大小

SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and< 1MB model size (2016)

作者：F. Iandola et al.

4.学习搭建问答神经网络

Learning to Compose Neural Networks for Question Answering (2016)

作者：J. Andreas et al.

5.用深度学习和大规模数据搜集，学习眼手协调的机器人抓取

Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection (2016)(Google)

作者：S. Levine et al.

6.将人隔离在外：贝叶斯优化算法回顾

Taking the human out of the loop: A review of bayesian optimization (2016)

作者：B. Shahriari et al.

7.Eie：压缩神经网络的高效推理引擎

Eie: Efficient inference engine on compressed deep neural network (2016)
作者：S. Han et al.

8.循环神经网络的自适性计算时间

Adaptive Computation Time for Recurrent Neural Networks (2016)

作者：A. Graves

9.像素循环神经网络

Pixel Recurrent Neural Networks (2016)

作者：A. van den Oord et al. (DeepMind)

10.LSTM：一场搜索空间的奥德赛之旅

LSTM: A search space odyssey (2016)

作者：K. Greff et al.

---

备注：本博客的所有论文下载链接: http://pan.baidu.com/s/1pL1UWRp
密码: 3kfj