【深度学习学习笔记】2.线性回归之一：最小二乘实现模型优化

原理：

最小二乘法即使经验风险最小化。
用平方损失函数衡量真实标签和预测标签的差异，经验风险函数$R(x)=\frac{1}{2}||y-X^{T}w|{|}^2$,$y$是真实标签，$X^{T}w$得到预测值。其对w的偏导数为$\frac{\partial R(w)}{\partial w}=-X(y-X^{T}w)$。令
$\frac{\partial R(w)}{\partial w}=0$得到最优的参数为$w^{\ast }=(X{X}^T{)}^{-1}Xy$

特征矩阵X，标签y

数据集下载

参数优化公式：

$w^{\ast }=(X{X}^T{)}^{-1}Xy$

代码

import numpy as np
import matplotlib.pyplot as plt

def readdata(filename):
    # filename = './train.txt'
    with open(filename, "r") as f:  # 打开文件
        data = f.readlines()  # 读取文
    X = []
    Y = []
    for line in data:
        list = line.split()
        X.append(float(list[0]))
        Y.append(float(list[1]))
    X = np.array(X)
    Y = np.array(Y)
    return X, Y

最小二乘法核心代码

def Leastsq(X, Y):
    # w∗ = (X·X转置)逆· X· y,
    # 其中X矩阵是n个样本点矩阵下面新增一行1
    array_1 = np.ones(len(X))
    X = np.vstack((X.T, array_1.T))

    w = np.dot(np.dot(np.linalg.inv(np.dot(X, X.T)), X), Y)

    return w

def main(X,Y):
    # train_file = './train.txt'
    # X, Y = readdata(train_file)

    w = Leastsq(X, Y)
    k = w[0]
    b = w[1]
    def f(x):
        y = k * x + b
        return y
    return f

def evaluate(ys, ys_pred):
    """评估模型。"""
    std = np.sqrt(np.mean(np.abs(ys - ys_pred) ** 2))
    return std

# 程序主入口（建议不要改动以下函数的接口）
if __name__ == '__main__':
    train_file = 'train.txt'
    test_file = 'test.txt'
    # 载入数据
    x_train, y_train = readdata(train_file)
    x_test, y_test = readdata(test_file)
    print(x_train.shape)
    print(x_test.shape)

    # 使用线性回归训练模型，返回一个函数f()使得y = f(x)
    f = main(x_train, y_train)

    y_train_pred = f(x_train)
    std = evaluate(y_train, y_train_pred)
    print('训练集预测值与真实值的标准差：{:.1f}'.format(std))

    # 计算预测的输出值
    y_test_pred = f(x_test)
    # 使用测试集评估模型
    std = evaluate(y_test, y_test_pred)
    print('预测值与真实值的标准差：{:.1f}'.format(std))

结果可视化

   
    plt.plot(x_train, y_train, 'ro', markersize=3)
    plt.plot(x_test, y_test)
    plt.plot(x_test, y_test_pred, 'k')
    plt.xlabel('x')
    plt.ylabel('y')
    plt.title('Linear Regression')
    plt.legend(['train', 'test', 'pred'])
    plt.show()

运行结果：

(300,)
(200,)
训练集预测值与真实值的标准差：2.0
预测值与真实值的标准差：2.2