Stanford Deep Learning wiki: http://deeplearning.stanford.edu/wiki/index.php/Main_Page
UFLDL Tutorial(英文):http://deeplearning.stanford.edu/wiki/index.php/UFLDL_Tutorial
http://ufldl.stanford.edu/tutorial/index.php/UFLDL_Tutorial
http://ufldl.stanford.edu/tutorial/
UFLDL Tutorial(中文):http://deeplearning.stanford.edu/wiki/index.php/UFLDL%E6%95%99%E7%A8%8B
UFLDL Tutorial Exercise code:https://github.com/dkyang/UFLDL-Tutorial-Exercise
Andrew Ng 教授的两门公开课:
Deep Learning:http://openclassroom.stanford.edu/MainFolder/CoursePage.php?course=DeepLearning
Machine Learning:http://openclassroom.stanford.edu/MainFolder/CoursePage.php?course=MachineLearning
Jeff Dean 2013 @ Stanford
http://i.stanford.edu/infoseminar/dean.pdf
一个对DL能干什么的入门级介绍,主要涉及Google在语音识别、图像处理和自然语言处理三个方向上的一些应用。参《Spanner and Deep Learning》(2013-01-19)
Hinton 2009
A tutorial on Deep Learning
Slides http://videolectures.net/site/normal_dl/tag=52790/jul09_hinton_deeplearn.pdf
Video http://videolectures.net/jul09_hinton_deeplearn/ (3 hours)
从神经网络的背景来分析DL,为什么要有DL说得很清楚。对DL的基本模型结构也说得很清楚。十分推荐
更多Hinton的教程 http://www.cs.toronto.edu/~hinton/nntut.html
斯坦福的Deep Learning公开课(2012)
Samy Bengio, Tom Dean and Andrew Ng
http://openclassroom.stanford.edu/MainFolder/CoursePage.php?course=DeepLearning
教学语言是Matlab。
参2011年的课程CS294A/CS294W Deep Learning and Unsupervised Feature Learning
更多的斯坦福工作: Deep Learning in Natural Language Processing
NIPS 2009 tutorial
这个介绍了DL在三个方向上的应用:tagging (parsing), semantic search, concept labeling
Ronan Collobert的Senna是一个c的深度学习实现,只有2000多行代码
ACL 2012 tutorial
Deep Learning for NLP (without Magic)
Video: http://www.youtube.com/watch?v=IF5tGEgRCTQ&list=PL4617D0E28A5781B0
Kai Yu’s Tutorial
Slides link: http://pan.baidu.com/share/link?shareid=136269&uk=2267174042
Video link: KaiYu_report.mp4 (519.2 MB)
Theano Deep Learning Tutorial
这个是实战, 如何用Python实现深度学习
http://deeplearning.net/tutorial/
Survey Papers
很多,不过初学看这两篇应该就够了
Yoshua Bengio, Aaron Courville, Pascal Vincent. (2012) Representation Learning: A Review and New Perspectives
Yoshua Bengio (2009). Learning Deep Architectures for AI.
更多
- Itamar Arel, Derek C. Rose, and Thomas P. Karnowski. (2010) Deep Machine Learning – A New Frontier in Artificial Intelligence Research 这篇没什么公式,也不长,就是笼统的介绍一下
- 截至2009的一些重要文章http://www.iro.umontreal.ca/~lisa/twiki/bin/view.cgi/Public/ReadingOnDeepNetworks
Deep Learning工具—— Theano
Theano是deep learning的Python库,要求首先熟悉Python语言和numpy,建议读者先看Theano basic tutorial,然后按照Getting Started 下载相关数据并用gradient descent的方法进行学习。
学习了Theano的基本方法后,可以练习写以下几个算法:
有监督学习:
- Logistic Regression - using Theano for something simple
- Multilayer perceptron - introduction to layers
- Deep Convolutional Network - a simplified version of LeNet5
无监督学习:
- Auto Encoders, Denoising Autoencoders - description of autoencoders
- Stacked Denoising Auto-Encoders - easy steps into unsupervised pre-training for deep nets
- Restricted Boltzmann Machines - single layer generative RBM model
- Deep Belief Networks - unsupervised generative pre-training of stacked RBMs followed by supervised fine-tuning
deeplearning tutorials:
http://deeplearning.net/tutorials/
http://www.iro.umontreal.ca/~bengioy/papers/ftml_book.pdf
Restricted boltzmann machine:
http://deeplearning.net/tutorial/rbm.html
http://blog.echen.me/2011/07/18/introduction-to-restricted-boltzmann-machines/
http://www.csri.utoronto.ca/~hinton/absps/guideTR.pdf
概率统计教程:
http://www.jdl.ac.cn/user/lyqing/StatLearning/StatlLearning_handout.html
Imagenet classification with deep convolutional neural networks代码:
https://code.google.com/p/cuda-convnet/