梯度下降(Gradient Descent)数学推导，多变量

下标及其表示

Notation	Size $x_1$	Number of bed rooms $x_2$	Number of floors $x_3$	Years $x_4$	Price $y$
$x^{(1)}=1^{th}$ $training$ $example$	2104	5	1	10	460
$x^{(2)}=2^{nd}$ $training$ $example$	1416	3（$x_2^{(2)}$）	2	8	232
$x^{(3)}=3^{rd}$ $training$ $example$	1534	3	2	5	315
$\cdots$	$\cdots$	$\cdots$	$\cdots$	$\cdots$	$\cdots$

$n$ = number of features = 4
$x^{(i)}$ = input of $i^{th}$ trainning example， 第$i$个训练数据，$4\times 1$ 向量，定义成列向量
$$x^{(2)}=\left(\begin{array}{c}1416\\ 3\\ 2\\ 8\end{array}\right)$$
$x_j^{(i)}$ = value of feature $j$ in $i^{th}$ trainning example, 标量

多变量表示

$h_{\theta }(x)(假设)={\theta }_0+{\theta }_1{x}_1+{\theta }_2{x}_2+\cdots$
定义：$x_0=1$
$$x=\left(\begin{array}{c}{x}_0\\ x_1\\ ⋮\\ x_n\end{array}\right)$$
$$\theta =\left(\begin{array}{c}{\theta }_0\\ {\theta }_1\\ ⋮\\ {\theta }_n\end{array}\right)$$
$$h_{\theta }(x)={\theta }^T\cdot x=\left(\begin{array}{cccc}{\theta }_0& {\theta }_1& \cdots & {\theta }_n\end{array}\right)\cdot \left(\begin{array}{c}{x}_0\\ x_1\\ ⋮\\ x_n\end{array}\right)$$

梯度下降

Hypothesis: $h_{\theta }(x)={\theta }^T\cdot x={\theta }_0{x}_0+{\theta }_1{x}_1+{\theta }_2{x}_2+\cdots$
Parameters: $\theta$ which is a $(n+1)\times 1$ vector
Cost function: $$J(\theta )=\frac{1}{2m}\sum _{i=1}^m(h_{\theta }(x^{(i)})-y^i{)}^2$$
Gradient Descent:

Repeat:{
${\theta }_j:={\theta }_j-\alpha \frac{\partial }{\partial {\theta }_j}J(\theta )$
}
So,
for j=0:
$$\frac{\partial }{\partial {\theta }_0}J(\theta )=\frac{1}{m}\sum _{i=1}^m(h_{\theta }(x^{(i)})-y^i)\frac{\partial (h_{\theta }(x^{(i)})-y^i)}{\partial {\theta }_0}$$
$$=\frac{1}{m}\sum _{i=1}^m(h_{\theta }(x^{(i)})-y^i)\frac{\partial ({\theta }_0{x}_0^{(i)}+{\theta }_1{x}_1^{(i)}+{\theta }_2{x}_2^{(i)}+\cdots )}{\partial {\theta }_0}$$
$$=\frac{1}{m}\sum _{i=1}^m(h_{\theta }(x^{(i)})-y^i)x_0^{(i)}$$
$$=\frac{1}{m}\sum _{i=1}^m(h_{\theta }(x^{(i)})-y^i)$$
for j=1:
$$\frac{\partial }{\partial {\theta }_1}J(\theta )=\frac{1}{m}\sum _{i=1}^m(h_{\theta }(x^{(i)})-y^i)\frac{\partial (h_{\theta }(x^{(i)})-y^i)}{\partial {\theta }_1}$$
$$=\frac{1}{m}\sum _{i=1}^m(h_{\theta }(x^{(i)})-y^i)\frac{\partial ({\theta }_0{x}_0^{(i)}+{\theta }_1{x}_1^{(i)}+{\theta }_2{x}_2^{(i)}+\cdots )}{\partial {\theta }_1}$$
$$=\frac{1}{m}\sum _{i=1}^m(h_{\theta }(x^{(i)})-y^i)x_1^{(i)}$$

So:
Repeat:{
$${\theta }_j:={\theta }_j-\alpha \frac{1}{m}\sum _{i=1}^m(h_{\theta }(x^{(i)})-y^i)x_j^{(i)}$$
}