线性回归模型求解推导

线性回归模型推导
线性拟合模型:
$$\begin{array}{rl}{h}_{\theta }(x)& ={\theta }_0+{\theta }_1{x}_1+{\theta }_2{x}_2\\ h_{\theta }(x)& =\sum _{i=0}^n{\theta }_i{x}_i={\theta }^{T}x\cdots ①\end{array}$$

误差，真实值和预测值之间存在的差异ε
对于每个样本：
$$y_i={\theta }^T{x}_i+{\epsilon }_i\cdots ②$$

假设：误差${\epsilon }_i$是独立并肯有相同的分布，并且服从均值为0方差为${\theta }^2$的高斯分布

预测值与误差由于服从高斯分布：
$$p({\epsilon }_i)=\frac{1}{\sqrt{2\pi }\sigma }exp(-\frac{({\epsilon }_i{)}^2}{2{\sigma }^2})$$

由①代入②式：
$$p(y_i|x_i;\theta )=\frac{1}{\sqrt{2\pi }\sigma }exp(-\frac{(y_i-{\theta }^T{x}_i{)}^2}{2{\sigma }^2})$$

使用似然函数，求解最优参数
似然函数：
$$L(\theta )=\prod _{i=1}^{m}p(y_i|x_i;\theta )=\prod _{i=1}^m\frac{1}{\sqrt{2\pi }\sigma }exp(-\frac{(y_i-{\theta }^T{x}_i{)}^2}{2{\sigma }^2})$$

函数变换为对数似然：
$$\begin{array}{rl}logL(\theta )& =log\prod _{i=1}^m\frac{1}{\sqrt{2\pi }\sigma }exp(-\frac{(y_i-{\theta }^T{x}_i{)}^2}{2{\sigma }^2})\\ & =\sum _{i=1}^{m}log\frac{1}{\sqrt{2\pi }\sigma }-\frac{1}{{\sigma }^2}\frac{1}{2}\sum _{i=1}^m(y_i-{\theta }^T{x}_i{)}^2\end{array}$$

目标：求似然函数的最大值,最小二乘法
领：
$$J(\theta )=\frac{1}{2}\sum _{i=1}^m(y_i-{\theta }^T{x}_i{)}^2$$
化简
$$J(\theta )=\frac{1}{2}\sum _{i=1}^m(h_{\theta }(x_i)-y_i)=\frac{1}{2}(X\theta -y{)}^T(X\theta -y)$$

求偏导：
$$\begin{array}{rl}{▽}_{\theta }J(\theta )& ={▽}_{\theta }(\frac{1}{2}(X\theta -y{)}^T(X\theta -y))\\ & ={▽}_{\theta }(\frac{1}{2}({\theta }^T{X}^T-y^T)(X\theta -y))\\ & ={▽}_{\theta }(\frac{1}{2}({\theta }^T{X}^{T}X\theta -{\theta }^T{X}^{T}y-y^{T}X\theta +y^{T}y))\\ & =\frac{1}{2}(2X^{T}X\theta -X^{T}y-(y^{T}X{)}^T)\\ & =X^{T}X\theta -X^{T}y\end{array}$$

当偏导为0时最小，$$\theta =(X^{T}X{)}^{-1}{X}^{T}y$$

评估方法
最常用的评估项$R^2$:$1-\frac{残差平方和}{总方差项}$ 解释因变量的度量值
$$1-\frac{\sum _{i=1}^m({\hat{y}}_i-y_i{)}^2}{{\sum }_{i=1}^m(y_i-\text{yˉ}{)}^2}$$