梯度下降
1、梯度下降最好是实现同步梯度下降,异步梯度下降的结果比较奇怪,但也可能有效;
2、If α is too small, gradient descent can be slow.
If α is too large, gradient descent can overshoot the minimum. It may fail toconverge, or even diverge.
3、As we approach a local minimum, gradient descent will automatically take smaller steps. So, no need to decreaseα over time.
当我们接近局部最小时,梯度下降算法会自动减小步长。所以没有必要减小α。
4、tricks:梯度下降需要将每一维的特征值缩放,均值归一化(mean normalization)
5、线性回归可以用梯度下降,也可以令导数为零求得
Gradient Descent
Normal Equation
• Need to choose α .• Needs many iterations.• Works well even when n is large. • No need tochoose α .• Don’t need to iterate.• Need to compute (X -1X) -1• Slow if n is very large.