Deep Learning训练经验-根据实验，持续更新

2020/05/14

Three key points of Neural Network：DataSet，Network，Loss.

The component that works as engine is the training method! In other word, the method while using to minimize the loss function.

最近做实验，训练小数据集，发现，根本不需要调learning rate 和 decay，用keras包自己带的“Adam” and "SGD"就够用了，而且BatchNormalization都可以在小数据集上不使用。

BN的优势在大数据集上，可以有效避免训练过程中loss的不收敛。

----to be continued!