机器学习笔记（3）-梯度下降法求多变量线性回归

数据生成

通过下式产生数据，作为训练集
$$y=1+rand+4\ast x_1+3\ast x_2+2\ast x_3+x_4$$
加入0~1之间的随机数使数据更加接近实际数据

假设

我们假设一个函数用于拟合上述数据
$$h(x)={\theta }_0+{\theta }_{1}x+{\theta }_2{x}_2+{\theta }_3{x}_3+{\theta }_4{x}_4$$
令
$$\begin{gathered} \Theta =\begin{array}{c}[{\theta }_0,{\theta }_1,{\theta }_2,{\theta }_3,{\theta }_4]\end{array} \\ X=\begin{array}{c}[one,x_1,x_2,x_3,x_4{]}^T\end{array} \\ h(x)=\Theta \ast X \end{gathered}$$

代价函数

$$J(\Theta )=\frac{1}{2m}\sum _{i=1}^m(h(x^{(i)})-y^{(i)}{)}^2$$
$m$为数据个数

梯度下降法

$${\theta }_i^k={\theta }_i^{k-1}-\alpha \frac{\partial J(\Theta )}{\partial {\theta }_i}$$
$\alpha$为学习率，用于控制梯度下降得快慢
这个式子表示新的${\theta }_i$等于上一个${\theta }_i$减去$\alpha \frac{\partial J(\Theta )}{\partial {\theta }_i}$使得${\theta }_i$始终朝着使代价函数$J(\Theta )$下降得方向变化
使用梯度下降法循环100次，或者使$J(\Theta )$的值下降到可以容许的误差范围内
$\frac{\partial J(\Theta )}{\partial {\theta }_i}$的偏导结果
$\frac{\partial J(\Theta )}{\partial {\theta }_0}=\frac{1}{m}{\sum }_{i=1}^m((h(x^{(i)})-y^{(i)})\cdot 1)$
$\frac{\partial J(\Theta )}{\partial {\theta }_1}=\frac{1}{m}{\sum }_{i=1}^m((h(x^{(i)})-y^{(i)})\cdot x_1)$
$\frac{\partial J(\Theta )}{\partial {\theta }_2}=\frac{1}{m}{\sum }_{i=1}^m((h(x^{(i)})-y^{(i)})\cdot x_2)$
$\frac{\partial J(\Theta )}{\partial {\theta }_3}=\frac{1}{m}{\sum }_{i=1}^m((h(x^{(i)})-y^{(i)})\cdot x_3)$
$\frac{\partial J(\Theta )}{\partial {\theta }_4}=\frac{1}{m}{\sum }_{i=1}^m((h(x^{(i)})-y^{(i)})\cdot x_4)$
令$D=h{x}^{(i)})-y^{(i)}$
${\Theta }^k={\Theta }^{k-1}-\alpha \frac{1}{m}{\sum }_{i=1}^m(D.\ast X)$

代码

%数据的产生,并加入噪声
close all;
clear;
x1 = 0:0.01:1;
x2 = 0:0.01:1;
x3 = 0:0.01:1;
x4 = 0:0.01:1;
%x的数据最大与最小之间倍数不宜过大，否则梯度下降法会失效引起代价函数迅猛增加
m=length(x1);
y = 1+rand(1,m)+4*x1+3*x2+2*x3+x4;

%假设函数为h(x) = theata0 + theata1*x1+ theata2*x2 + theata3*x3+theata4*x4;
theata = rand(1,5);%随机产生系数
alpha = 0.1;%学习率
X=[ones(1,m);x1;x2;x3;x4];
var = zeros(1,10);
for i=1:100%迭代1000次
h_theata = theata*X;%假设函数
d=h_theata - y;%假设函数与原始数据间的差
var(i) =  1/(2*m)*sum((d).^2);%方差
%梯度下降法迭代求解新系数，代价函数1/2m *sum((h(x) - y)^2)
theata = theata -alpha*1/m*sum(transpose(d.*X));
end
%代价函数1/2m *sum((h(x) - y)^2)的变化曲线
plot(1:i,var);
title('代价函数大小变化');