多元线性回归算法python实现_机器学习算法python实现-线性回归

线性回归-Linear Regreesion

给定数据集

线性回归试图学得

，使

与

之间的差别尽可能小。

如何确定

和

，关键在于如何衡量

与

之间的差别,可以通过均方误差最小化

基于均方误差最小化来进行模型求解的方法称为“最小二乘法”

这里

是关于w和b的凸函数，当它关于w和b的导数均为零时，得到w和b的最优解。

求w和b的偏导：

令偏导为0：

推广到矩阵形式w的梯度为：

线性回归、批梯度下降、随机梯度下降优化具体算法可参考：kamidox.com​blog.kamidox.com

1.一元线性回归

import numpy as np

import matplotlib.pyplot as plt

x=np.array([1,2,3,4,5],dtype=np.float)

y=np.array([1,3.0,2,3,5],dtype=np.float)

plt.scatter(x,y)

x_mean=np.mean(x)

y_mean=np.mean(y)

num=0.0

d=0.0

for x_i,y_i in zip(x,y):

num+=(x_i-x_mean)*(y_i-y_mean)

d+=(x_i-x_mean)**2

a=num/d

b=y_mean-a*x_mean

y_hat=a*x+b

plt.figure(2)

plt.scatter(x,y) #描绘散点

plt.plot(x,y_hat,c='r') #描绘直线

2.多元线性回归批梯度下降

from sklearn.datasets import load_diabetes

from sklearn.model_selection import train_test_split

diabetes = load_diabetes()

data = diabetes.data

target = diabetes.target.reshape((-1,1))

X_train, X_test, y_train, y_test = train_test_split( data, target, test_size=0.3, random_state=2020)

print('X_train=', X_train.shape)

print('X_test=', X_test.shape)

print('y_train=', y_train.shape)

print('y_test=', y_test.shape)

out:

X_train= (309, 10)

X_test= (133, 10)

y_train= (309, 1)

y_test= (133, 1)

模型主体：

def linar_train(X, y, learning_rate=0.01, epochs=30000):

w, b = initialize(X.shape[1]) #初始化参数

for i in range(1, epochs):

y_hat, loss, dw, db = linar_loss(X, y, w, b)

w += -learning_rate * dw

b += -learning_rate * db

if i % 10000 == 0:

print('epoch %d loss %f' % (i, loss))

params = {

'w': w,

'b': b

}

grads = {

'dw': dw,

'db': db

}

return loss, params, grads

def initialize(dims):

"""将线性回归中w全部初始化为0w为dims行 1列的0矩阵"""

w = np.zeros((dims, 1))

b = 0

return w, b

def linar_loss(X, y, w, b):

"""计算损失函数num_train为数据集的数目num_feature为数据特征数y_hat为预测的y值loss 计算均方误差"""

num_train = X.shape[0]

num_feature = X.shape[1]

y_hat = np.dot(X, w) + b

loss = np.sum((y_hat-y)**2)/num_train #计算均方误差

dw = np.dot(X.T, (y_hat-y)) /num_train #X.T矩阵转置

db = np.sum((y_hat-y)) /num_train

return y_hat, loss, dw, db

训练数据：

loss, params, grads = linar_train(X_train, y_train, 0.001, 500000)

查看参数：

params

{'w': array([[ 18.12432661],

[-107.67245554],

[ 404.37142851],

[ 226.28953116],

[ -10.89733854],

[ -76.80225482],

[-174.88980099],

[ 126.33236945],

[ 353.19542078],

[ 170.24624517]]),

'b': 151.95335186539964}

可视化：

import matplotlib.pyplot as plt

f = X_test.dot(params['w']) + params['b']

plt.scatter(range(X_test.shape[0]), y_test)

plt.plot(f, color = 'darkorange')

plt.xlabel('X')

plt.ylabel('y')

plt.show()

参考：