神经网络学习率

在吴恩达作业，第一课第二周中，有这样一个例子

 

Interpretation:

• Different learning rates give different costs and thus different predictions results.
• If the learning rate is too large (0.01), the cost may oscillate up and down. It may even diverge (though in this example, using 0.01 still eventually ends up at a good value for the cost).
• A lower cost doesn't mean a better model. You have to check if there is possibly overfitting. It happens when the training accuracy is a lot higher than the test accuracy.
• In deep learning, we usually recommend that you:
  • Choose the learning rate that better minimizes the cost function.
  • If your model overfits, use other techniques to reduce overfitting. (We'll talk about this in later videos.)

 

总结：

学习率太大，可能震荡，无法逼近最优解

学习率太小，迭代次数多，计算量大，容易过拟合