机器学习之线性回归

简单线性回归算法

目标是找到a和b，使得${\sum }_{i=1}^m(y_i-a{x}_i-b{)}^2$尽可能的小。
$$a=\frac{{\sum }_{i=1}^m(x_i-\overline{x})(y_i-\overline{y})}{{\sum }_{i=1}^m(x_i-\overline{x}{)}^2}$$
$$b=\overline{y}-a\overline{x}$$

通过上面这个式子，可以自己设计一个求解线性回归的函数：

class SimpleLinearRegression:
    def __init__(self):
        self.a_ = None
        self.b_ = None
    
    def fit(self, x_train, y_train):
        # 这里只能处理一维的数据
        x_mean = np.mean(x_train)
        y_mean = np.mean(y_train)
        
        num = (x_train - x_mean).dot(y_train - y_mean) # 采用向量化的计算方式，效率有极大的提升
        d = (x_train - x_mean).dot(x_train - x_mean)
        
        self.a_ = num / d
        self.b_ = y_mean - self.a_ * x_mean
        return self
    
    def predict(self, x_predict):
        return np.array([self._predict(x) for x in x_predict])

    def _predict(self, x_single):
        return self.a_ * x_single + self.b_

对算法进行测试

import numpy as np
m = 100
x = np.random.random(size=m)
y = x*2 + 3 +np.random.normal(size=m)

reg = SimpleLinearRegression()
reg.fit(x, y)
reg.a_
>>>1.909471288723333
reg.b_
>>>3.0055997219091517

对结果绘图

import matplotlib.pyplot as plt
y_hat = reg.predict(x)
plt.scatter(x,y)
plt.plot(x, y_hat, color='r')
plt.show()

对回归算法进行评价

均方误差MSE（Mean Squared Error）$\frac{1}{m}{\sum }_{i=1}^m(y_i-\widehat{y_i}{)}^2$

from sklearn.metrics import mean_squared_error
mean_squared_error(y_test, y_hat)

均方根误差RMSE（Root Mean Squared Error） $\sqrt{MSE}$

平均绝对误差MAE（Mean Absolute Error) $\frac{1}{m}{\sum }_i^m|y_i-\widehat{y_i}|$

from sklearn.metrics import mean_absolute_error
mean_absolute_error(y_test, y_hat)

R squard $R^2=1-\frac{S{S}_{residual}}{S{S}_{total}}=1-\frac{{\sum }_i({\hat{y}}_i-y_i{)}^2}{{\sum }_i({\overline{y}}_i-y_i{)}^2}=1-\frac{{\sum }_i({\hat{y}}_i-y_i{)}^2/m}{{\sum }_i({\overline{y}}_i-y_i{)}^2/m}=1-\frac{MSE}{Var}$:
分子描述的是模型预测产生的错误。分母描述的$y=\overline{y}$产生的错误，就是假设无论x为多少，预测值都是$\overline{y}$时产生的错误，这也被称为基准。
R squared 越大越好，模型不犯任何错误，那么R squared就是1。如果R squared < 0, 说明训练的模型极差，升值不如基准。

多元线性回归

from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
from sklearn import datasets
boston = datasets.load_boston() # 加载波士顿房价的数据
x = boston.data
y = boston.target

x_train, x_test, y_train, y_test = train_test_split(x, y, random_state=100)

lin_reg = LinearRegression()
lin_reg.fit(x_train, y_train)
lin_reg.coef_ # 系数
lin_reg.intercept_ # 截距
lin_reg.score(x_test, y_test)