如何训练deep architectures

简介

如何训练深度网络

Deep learning methods aim at learning feature hierarchies with features from higher levels of the hierarchy formed by the composition of lower level features.
在抽象的多个层次自动学习特征允许系统能够学习复杂的函数直接映射输入数据至输出。对于高层次抽象十分重要。自动学习powerful features 十分重要，尤其是在数据量增大和机器学习连续成长的时候。
Depth of Architectures 涉及在学习方法中非线性变换成分的数量。神经网络的研究者们wanted for decades to train deep multi-layer neural
networks。但是2006年Hinton引入DBN(deep belief network)以及一个贪婪学习算法，在其中greedily trains one layer at a time, exploiting
an unsupervised learning algorithm for each layer, a Restricted Boltzmann Machine (RBM). 不久之后，auto-encoder被提出，利用了同样的方法：

guiding the training of intermediate levels of representation using unsupervised learning, which can be performed locally at each level.

deep networks have been applied with success not only in classification tasks, but also in regression, d