欢迎使用CSDN-markdown编辑器

Linear Regression with One Variable

trainning set

x	y
1	3
2	6
3	9
4	14

Trainning examples is 4 .
x is features
y is output

$h_\theta = \theta_0+\theta_1x$ is Hypothesis.
$\theta = \left[ \begin{matrix} \theta_0 \\ \theta_1 \\ \end{matrix} \right]$
$X = \left[ \begin{matrix} 1&1\\ 1&2\\ 1&3\\ 1&4\\ \end{matrix} \right] =\left[ \begin{matrix} x_0&x1\\ \end{matrix} \right] \in R^{m*{(n+1)}}$

m=4 is the number of trainning example
n = 1 is the numbei of feature

Hypothesis:
$h_\theta = \theta_0+\theta_1x$
Parameters:
$\theta_0 ,\theta_1$
cost Function:
$J(\theta)=J(\theta_0,\theta_1)=\frac{1}{2m}\sum_{i=1}^m (h_\theta(x^{(i)} )-y^{(i)})^2 = \frac{1}{2m}\sum_{i=1}^m ((\theta^T x^{(i)})-y^{(i)})^2 = \frac{1}{2m} (X\theta-y).^2$
Goal: $minimize J(\theta_0,\theta_1)$

Use Gradient descent algorithm minimize the cost function $J(\theta_0,\theta_1)$
repeat{
$\theta_0:= \theta_0-\alpha\frac{1}{m}\sum_{i=1}^m (h_\theta(x^{(i)} )-y^{(i)})=\theta_0-\alpha\frac{1}{m}\sum_{i=1}^m ((\theta^T x^{(i)})-y^{(i)})$

$\theta_1:= \theta_1-\alpha\frac{1}{m}\sum_{i=1}^m (h_\theta(x^{(i)} )-y^{(i)})x^{(i)}=\theta_1-\alpha\frac{1}{m}\sum_{i=1}^m ((\theta^T x^{(i)})-y^{(i)})x^{(i)}=\theta_1 -\alpha\frac{1}{m}((X\theta-y)^TX(:,2))$
}
向量形式
repeat{
$\theta= \theta -\alpha\frac{1}{m}((X\theta-y)^TX)$
}