Deep Learning 资料汇总_deep learning wiki-CSDN博客

In the past 30 years, tremendous progress has been made in building effective classification models. Despite the success, we have to realize that, in major AI challenges, the key bottleneck is not the quality of classifiers but that of features. Since 2006, learning high-level features using deep architectures has become a big wave of new learning paradigms. In recent two years, performance breakthrough was reported in both image and speech recognition tasks, indicating deep learning are not something ignorable. In this talk, I will walk through the recent works and key building blocks, e.g., sparse coding, RBMs, auto-encoders, etc. and list the major research topics, including modeling and computational issues. In the end, I will discuss what might be interesting topics for future research.

Bio of Dr. Kai Yu:

余凯任百度技术副总监，多媒体部负责人，主要负责公司在语音，图像，音频等领域面向互联网和移动应用的技术研发。加盟百度前，余凯博士在美国NEC研究院担任Media Analytics部门主管(Department Head)，领导团队在机器学习、图像识别、多媒体检索、视频监控，以及数据挖掘和人机交互等方面的产品技术研发。此前他曾在西门子公司任Senior Research Scientist。2011年曾在斯坦福大学计算机系客座主讲课程“CS121: 人工智能概论”。他在NIPS, ICML, CVPR, ICCV, ECCV，SIGIR, SIGKDD，TPAMI，TKDE等会议和杂志上发表了70多篇论文，H-index=28，曾担任机器学习国际会议ICML10, ICML11, NIPS11, NIPS12的Area Chair. 2012年他被评为中关村高端领军人才和北京市海聚计划高层次海外人才。

Slides link: http://pan.baidu.com/share/link?shareid=136269&uk=2267174042

Video link: KaiYu_report.mp4 (519.2 MB)

Theano Deep Learning Tutorial

这个是实战, 如何用Python实现深度学习

http://deeplearning.net/tutorial/

Code https://github.com/lisa-lab/DeepLearningTutorials

Survey Papers

很多，不过初学看这两篇应该就够了

Yoshua Bengio, Aaron Courville, Pascal Vincent. (2012) Representation Learning: A Review and New Perspectives

Yoshua Bengio (2009). Learning Deep Architectures for AI.

Itamar Arel, Derek C. Rose, and Thomas P. Karnowski. (2010) Deep Machine Learning – A New Frontier in Artificial Intelligence Research 这篇没什么公式，也不长，就是笼统的介绍一下
截至2009的一些重要文章http://www.iro.umontreal.ca/~lisa/twiki/bin/view.cgi/Public/ReadingOnDeepNetworks

Deep Learning工具—— Theano

Theano是deep learning的Python库，要求首先熟悉Python语言和numpy，建议读者先看Theano basic tutorial，然后按照Getting Started 下载相关数据并用gradient descent的方法进行学习。

学习了Theano的基本方法后，可以练习写以下几个算法：

有监督学习：

Logistic Regression - using Theano for something simple
Multilayer perceptron - introduction to layers
Deep Convolutional Network - a simplified version of LeNet5

无监督学习：

Auto Encoders, Denoising Autoencoders - description of autoencoders
Stacked Denoising Auto-Encoders - easy steps into unsupervised pre-training for deep nets
Restricted Boltzmann Machines - single layer generative RBM model
Deep Belief Networks - unsupervised generative pre-training of stacked RBMs followed by supervised fine-tuning