How to train your neural network?（如何训练你的神经网络）

最新推荐文章于 2022-12-12 16:28:00 发布

VOIX

最新推荐文章于 2022-12-12 16:28:00 发布

阅读量304

点赞数

分类专栏：自学

本文链接：https://blog.csdn.net/weixin_43874070/article/details/115626439

版权

本文介绍了如何训练神经网络，包括使用小批量数据进行梯度检查、监控训练和验证准确性、选择合适的更新方法如SGD+Momentum或Adam、逐步降低学习率以及通过随机搜索优化超参数。此外，还建议通过模型集成来提升性能。

摘要由CSDN通过智能技术生成

How To train a Neural Network:

Gradient check your implementation with a small batch of data and be aware of the pitfalls.
As a sanity check, make sure your initial loss is reasonable, and that you can achieve 100% training accuracy on a very small portion of the data
During training, monitor the loss, the training/validation accuracy, and if you’re feeling fancier, the magnitude of updates in relation to parameter values (it should be ~1e-3), and when dealing with ConvNets, the first-layer weights.
The two recommended updates to use are either SGD+Nesterov Momentum or Adam.
Decay your learning rate over the period of the training. For example, halve the learning rate after a fixed number of epochs, or whenever the validation accuracy tops off.
Search for good hyperparameters with random sea

关注