Neural Networks and Deep Learning 资料整理

CH1 Using neural nets to recognize handwritten digits

1. RNN的wiki：recurrent neural networks

2. Cauchy-Schwarz不等式的wiki：Cauchy-Schwarz inequality

CH2 How the backpropagation algorithm works

1. 提出backpropagation算法的paper：Learning representations by back-propagating errors

CH3 Improving the way neural networks learn

1.  Cross entropy的wiki：brief summary

2.  一个哲学问题的wiki：problem of induction

3. paper：Gradient-Based Learning Applied to Document Recognition

4. paper：ImageNet Classification with Deep Convolutional Neural Networks

5. paper：Improving neural networks by preventing co-adaptation of feature detectors

6. paper：Best Practices for Convolutional Neural Networks Applied to Visual Document Analysis

7. paper：Scaling to very very large corpora for natural language disambiguation

8. paper：Practical Recommendations for Gradient-Based Training of Deep Architectures

9. paper：Deep, Big, Simple Neural Nets Excel on Handwritten Digit Recognition

10. paper：Random search for hyper-parameter optimization

11. paper：Practical Bayesian optimization of machine learning algorithms

12. paper：Practical recommendations for gradient-based training of deep architectures

13. paper：Efficient BackProp

14. paper：Neural Networks: Tricks of the Trade

15. paper：On the importance of initialization and momentum in deep learning

16. wiki：L-BFGS

17. paper： Understanding the difficulty of training deep feedforward networks

18. paper：What is the Best Multi-Stage Architecture for Object Recognition?

19. paper：Deep Sparse Rectifier Neural Networks

20. paper：Rectified Linear Units Improve Restricted Boltzmann Machines

CH4 A visual proof that neural nets can compute any function

1. paper：Approximation by superpositions of a sigmoidal function

2. paper：Multilayer feedforward networks are universal approximators

CH5 Why are deep neural networks hard to train?

1. paper：On the correlation of parity and small-depth circuits 

2. paper：On the number of response regions of deep feed forward networks with piece-wise linear activations

3. paper： Learning deep architectures for AI 

4. paper：Gradient flow in recurrent nets: the difficulty of learning long-term dependencies

5. paper：Untersuchungen zu dynamischen neuronalen Netzen 

6. paper：Understanding the difficulty of training deep feedforward neural networks

CH6 Deep learning

1. paper：Visualizing and Understanding Convolutional Networks

2. DL相关网站： LeNet-5

3. Theano：Theano: A CPU and GPU Math Expression Compiler in Python

4. Classification datasets results，含有大量数据集和paper：informative summary page

5. paper：Building high-level features using large scale unsupervised learning

6. paper：ImageNet: a large-scale hierarchical image database

7. 图像数据集网站：ImageNet

8. paper：ImageNet classification with deep convolutional neural networks 

9. paper：Theano-based large-scale visual recognition with multiple GPUs

10. paper：Going deeper with convolutions

11. paper：ImageNet large scale visual recognition challenge

12. paper：Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks

13. paper：Intriguing properties of neural networks

14. paper：Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images

15. paper：A fast learning algorithm for deep belief nets

16. paper：Reducing the dimensionality of data with neural networks

17. paper：RNN A 2014 paper

18. paper：RNN A second paper, also from 2014

19. LSTMs：Hochreiter and Schmidhuber in 1997

20. paper：to recognize shapes, first learn to generate images.

21. DBNs：this overview

22. DBNs：this article

23. paper：natural language processing

24. paper：also this informative review paper

25. paper： music informatics

26. paper：play video games well   this followup

27. paper：annotate photos and videos