【计算机科学】【2013】采用Dropout提升神经网络的性能

本文为加拿大多伦多大学（作者：Nitish Srivastava）的硕士论文，共26页。

具有大量参数的深度神经网络是非常强大的机器学习系统。然而，在这种网络中，过拟合是一个严重的问题。大型网络的收敛速度也很慢，因此很难在测试时通过组合多个不同的大型神经网络来处理过拟合问题。Dropout是解决这个问题的一种技术，关键的想法是在训练过程中从神经网络中随机删除神经单元节点（以及与它们的连接），这就防止了单元之间的配合过多。在训练过程中，Dropout神经单元会造成网络深度变浅。可能变浅的网络数量是网络中神经单位数量的指数。在测试时，所有可能的变浅网络使用一个近似的模型平均过程进行组合。这种近似模型组合之后的Dropout训练显著减少了过度拟合，并比其它正则化方法有了重大改进。在这项工作中，我们描述了使用Dropout提高神经网络性能的模型，通常能够在基准数据集上获得最先进的测试结果。

Deep neural nets with a huge number ofparameters are very powerful machine learning systems. However, overfitting isa serious problem in such networks. Large networks are also slow to use, makingit difficult to deal with overfitting by combining many different large neuralnets at test time. Dropout is a technique for addressing this problem. The keyidea is to randomly drop units (along with their connections) from a neuralnetwork during training. This prevents the units from co-adapting too much.Dropping units creates thinned networks during training. The number of possiblethinned networks is exponential in the number of units in the network. At testtime all possible thinned networks are combined using an approximate modelaveraging procedure. Dropout training followed by this approximate modelcombination significantly reduces overfitting and gives major improvements overother regularization methods. In this work, we describe models that improve theperformance of neural networks using dropout, often obtaining state-of-the-artresults on benchmark datasets.

1 引言
2 带前馈神经网络的Dropout
3 带Boltzmann机的Dropout
4 边缘化Dropout
5 结论

下载英文原文地址：

http://page2.dfpan.com/fs/aldc4j92c2e102c9163/

更多精彩文章请关注微信号：