由上一篇文章我们知道，当 J ( θ 0 , θ 1 ) J(\theta_{0},\theta_{1}) J(θ0,θ1)最小时，参数 ( θ 0 , θ 1 ) (\theta_{0},\theta_{1}) (θ0,θ1)所构成的假设函数 h θ = θ 0 + θ 1 x h_{\theta}=\theta_{0}+\theta_{1}x hθ=θ0+θ1x最能够拟合样本数据.
我们以
θ
0
\theta_{0}
θ0为x轴，
θ
1
\theta_{1}
θ1为y轴，代价函数为Z轴画出
θ
0
\theta_{0}
θ0、
θ
1
\theta_{1}
θ1取不同值的图像：
We will know that we have succeeded when our cost function is at the very bottom of the pits in our graph, i.e. when its value is the minimum. The red arrows show the minimum points in the graph.
The way we do this is by taking the derivative (the tangential line to a function) of our cost function. The slope of the tangent is the derivative at that point and it will give us a direction to move towards. We make steps down the cost function in the direction with the steepest descent. The size of each step is determined by the parameter α, which is called the learning rate.
For example, the distance between each ‘star’ in the graph above represents a step determined by our parameter α. A smaller α would result in a smaller step and a larger α results in a larger step. The direction in which the step is taken is determined by the partial derivative of J ( θ 0 , θ 1 ) J(\theta_0,\theta_1) J(θ0,θ1)，Depending on where one starts on the graph, one could end up at different points. The image above shows us two different starting points that end up in two different places.
The gradient descent algorithm is:
θ
j
:
=
θ
j
−
α
∗
∂
J
(
θ
0
,
θ
1
)
∂
θ
j
\theta_j:=\theta_j-\alpha*\frac{\partial{J(\theta_0,\theta_1)}}{\partial{\theta_j}}
θj:=θj−α∗∂θj∂J(θ0,θ1)
repeat until convergence:
where
j=0,1 represents the feature index number.
At each iteration j, one should simultaneously update the parameters
θ
1
,
θ
2
,
.
.
.
,
θ
n
\theta_1, \theta_2,...,\theta_n
θ1,θ2,...,θn
Updating a specific parameter prior to calculating another one on the
j
(
t
h
)
j^{(th)}
j(th) iteration would yield to a wrong implementation.
扫一扫
851
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
