吴恩达·Machine Learning || chap2 Linear regression with one variable 简记

2 Linear regression with one variable

2-1 Model representation

Training set

m = Number of training examples

x’s = “input” variable/features

y’s = “output” variable/“target” variable

(x,y) = one training example

Hypothesis 假设函数
$$h_{\theta }(x)={\theta }_0+{\theta }_{1}x$$

2-2 Cost Function

Goal: minimize
$$\frac{1}{2m}\sum _1^m(h_{\theta }(x^{(i)}-y^{(i)}{)}^2$$

除以m是使得误差平均到每个样本，除以2是一个微积分技巧，用于消除计算偏导数时出现的2。​​

Cost Function(Square error function )
$$J({\theta }_0,{\theta }_1)=\frac{1}{2m}\sum _{i=1}^m(h_{\theta }(x^{(i)}-y^{(i)}{)}^2$$

2-3 Cost Function Intuition I

Simplified

${\theta }_0=0$

2-4 Cost Function Intuition II

Contour plot/figure 等高线图

2-5 Gradient descent

Outline

• Start with some ${\theta }_0,{\theta }_1$

• Keep changing ${\theta }_0,{\theta }_1$ to reduce $J({\theta }_0,{\theta }_1)$

  until we hopefully end up with a minimum

Gradient descent algorithm

repeat until convergence{

${\theta }_j:={\theta }_j-\alpha \frac{\partial }{\partial {\theta }_j}J({\theta }_0,{\theta }_1)$ （for j=0 and j=1）

}

$\alpha$ learning rate

Correct ：Simultaneous update

$temp0：$${\theta }_0:={\theta }_0-\alpha \frac{\partial }{\partial {\theta }_0}J({\theta }_0,{\theta }_1)$​​

$temp1：$ ${\theta }_1:={\theta }_1-\alpha \frac{\partial }{\partial {\theta }_1}J({\theta }_0,{\theta }_1)$

${\theta }_0:$=$temp0$

${\theta }_1$ :=$temp1$

2-6 Gradient descent Intuition

Learning rate $\alpha$

if $\alpha$ is too small,gradient descent can be slow

if $\alpha$ is too large,gradient descent can overshoot the minimum .It may fail to converge,or even diverge

learning rate $\alpha$​​ fixed

As we approach a local minimum, gradient descent will automatically take smaller steps. So, no need to decrease $\alpha$ over time

2-7 Gradient descent for linear regression

Gradient decent algorithm

repeat until convergence{
$$\{\begin{array}{l}{\theta }_0:={\theta }_0-\alpha \frac{1}{m}{\sum }_{i=1}^m(h_{\theta }(x^{(i)}-y^{(i)})\\ {\theta }_1:={\theta }_1-\alpha \frac{1}{m}{\sum }_{i=1}^m(h_{\theta }(x^{(i)}-y^{(i)}).x^{(i)}\end{array}$$
}

update ${\theta }_0$ and ${\theta }_1$​ simultaneously

convex function 凸函数

”Batch“ Gradient Descent

Batch ": Each step of gradient descent uses all the training examples