[paper] Dropout

Dropout

Abstract

过拟合是指在训练集上表现很好，但是在测试集上表现不好

Overfitting can be reduced by using “dropout” to prevent complex co-adaptations on the training data.
可以通过使用dropout来避免在训练集上co-adaptations以减少过拟合

On each presentation of each training case, each hidden unit is randomly omitted from the network with a probability of 0.5, so a hidden unit cannot rely on other hidden units being present.
dropout是在每一个隐藏层上，随机忽略50%的隐藏单元

在测试的时候，我们使用"mean network"（包含所有隐藏单元，但是权重减半）来弥补之前的随机忽略

在全连接层中，所有隐藏层都有dropout比只有一个隐藏层有dropout好

在输入中添加dropout也是有帮助的