吴恩达机器学习笔记2-Linear_Regression_with_One_Variable

最新推荐文章于 2022-01-06 20:57:52 发布

嘟嘟鱼小诺

最新推荐文章于 2022-01-06 20:57:52 发布

阅读量276

点赞数

分类专栏：机器学习

本文链接：https://blog.csdn.net/duduyuxiaonuo/article/details/79003254

版权

机器学习专栏收录该内容

2 篇文章 0 订阅

订阅专栏

监督学习之回归问题——预测房屋售价

数据集

Training set of housing prices：

Size in $feet^2(x)$	Price(￥) in ${1000}'s(y)$
2014	460
1416	232
1534	315
852	178
$\cdots$	$\cdots$

注释：

$m:训练集大小$

${x}'s = "input" variable/feature$

${y}'s = "output" variable /"target" variable$

$(x,y) 是一个训练样本$

$(x^{(i)},y^{(i)},第i个训练样本$

线性回归模型：

h θ (x) = θ 0 + θ 1 x

$h_\theta (x)= \theta_0 + \theta_1x$

θi模型参数 $\theta_i模型参数$

cost function

在线性回归中，我们有一个数据集，我们要做的就是得出模型参数的值，尽量地与这些数据点很好的拟合。

m i n i m i z e θ 0, θ 1 1 2 m \sum i = 1 m (h θ (x (i)) - y (i)) 2

$\underset{\theta_0, \theta_1 }{minimize}\frac{1}{2m}\sum\limits_{i=1}^{m}(h_\theta(x^{(i)})-y^{(i)})^2$

令J(θ0,θ1)=12m∑i=1m(hθ(x(i))−y(i))2，求minθ0,θ1J(θ0,θ1) $令J(\theta_0,\theta_1)=\frac{1}{2m}\sum\limits_{i=1}^{m}(h_\theta(x^{(i)})-y^{(i)})^2，求\underset{\theta_0, \theta_1 }{min}J(\theta_0,\theta_1)$

平方误差代价函数

是	一般	简化
模型	$h_\theta (x)= \theta_0 + \theta_1x$	$h_\theta (x)= \theta_1x,\theta_0=0$
参数	$\theta_0,\theta_1：不同的参数不同的模型$	$\theta_1$
代价函数	$J(\theta_0,\theta_1)=\frac{1}{2m}\sum\limits_{i=1}^{m}(h_\theta(x^{(i)})-y^{(i)})^2$	$J(\theta_1)=\frac{1}{2m}\sum\limits_{i=1}^{m}(h_\theta(x^{(i)})-y^{(i)})^2$
目标	$\underset{\theta_0, \theta_1 }{min}J(\theta_0,\theta_1)$	$\underset{\theta_1}{min}J(\theta_1)$

$两个参数的\underset{\theta_0, \theta_1}{min}J(\theta_0,\theta_1)图形是一个 3D的图形，但我们不会使用3D曲面来展示代价函数J，而是使用等高线图，或者成为等高图像$

Gradient Descent梯度下降法

梯度下降法步骤：
Have some function：

$J(\theta_0,\theta_1)$

Want:

$\underset{\theta_0, \theta_1 }{min}J(\theta_0,\theta_1)$

outLine
- Keep changing $\theta_0, \theta_1$ to reduce $J(\theta_0,\theta_1)$ until we hopefully end up at a minimum
- start with some $\theta_0, \theta_1$ (say $\theta_0=0, \theta_1=0$ )

梯度下降法求出的”最优解“只是局部最优解。
梯度下降法算法：
repeat until converage{

θ j : = θ j - α \partial \partial θ j J (θ 0, θ 1)

$\theta_j := \theta_j - \alpha \frac{\partial }{\partial \theta_j}J(\theta_0,\theta_1)$
}

\partial \partial θ j J (θ 0, θ 1) = \partial \partial θ j 1 2 m \sum i = 1 m (h θ (x (i)) - y (i)) 2 = \partial \partial θ j 1 2 m \sum i = 1 m (θ 0 + θ 1 x (i) - y (i)) 2 θ 0 j = 0 : \partial \partial θ 0 J (θ 0, θ 1) = 1 m \sum i = 1 m (h θ (x (i)) - y (i)) θ 1 j = 1 : \partial \partial θ 1 J (θ 0, θ 1) = 1 m \sum i = 1 m (h θ (x (i)) - y (i)) \cdot x (i)

$\frac{\partial }{\partial \theta_j}J(\theta_0,\theta_1)=\frac{\partial }{\partial \theta_j}\frac{1}{2m}\sum\limits_{i=1}^{m}(h_\theta(x^{(i)})-y^{(i)})^2 \\ =\frac{\partial }{\partial \theta_j}\frac{1}{2m}\sum\limits_{i=1}^{m}(\theta_0+\theta_1x^{(i)}-y^{(i)})^2 \\ \theta_0 \quad j=0: \quad \frac{\partial }{\partial \theta_0}J(\theta_0,\theta_1) = \frac{1}{m}\sum\limits_{i=1}^{m}(h_\theta(x^{(i)})-y^{(i)}) \\ \theta_1\quad j=1:\quad \frac{\partial }{\partial \theta_1}J(\theta_0,\theta_1) = \frac{1}{m}\sum\limits_{i=1}^{m}(h_\theta(x^{(i)})-y^{(i)})\cdot x^{(i)}$