本节讨论下线性回归模型的性能评价指标
对于机器学习的两个基本问题分类和回归的评价方式有所不同,分类问题一般通过分类准确率、召回率、F1值、ROC/AUC等手段进行模型的评估。对于回归问题,该如何评价? 这里简要列举部分评估方法。
1、残差估计
总体思想是计算实际值与预测值间的差值简称残差。从而实现对回归模型的评估,一般可以画出残差图,进行分析评估、估计模型的异常值、同时还可以检查模型是否是线性的、以及误差是否随机分布。
import matplotlib.pyplot as plt
import seaborn as sns
import pandas as pd
import numpy as np
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
# 数据读取
df = pd.read_csv('dataset/boston.csv', sep=',')
df.columns = ['CRIM', 'ZN', 'INDUS', 'CHAS',
'NOX', 'RM', 'AGE', 'DIS', 'RAD',
'TAX', 'PTRATIO', 'LSTAT', 'MEDV']
# 所有属性拟合线性模型
X = df.iloc[:, :-1].values
y = df[['MEDV']].values
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.3, random_state=0)
# 模型训练
slr = LinearRegression()
slr.fit(X_train, y_train)
y_train_pred = slr.predict(X_train)
y_test_pred = slr.predict(X_test)
# 残差评估方法
plt.scatter(y_train_pred, y_train_pred-y_train,
c='blue', marker='o', label='Training data')
plt.scatter(y_test_pred, y_test_pred-y_test,
c='lightgreen', marker='s', label='Test data')
plt.xlabel('Predicted values')
plt.ylabel('Residuals')
plt.legend(loc='upper left')
plt.hlines(y=0, xmin=-10, xmax=50, lw=2, colors='red')
plt.xlim([-10, 50])
plt.savefig('result/residuals_metric.png')
plt.show()
效果图
可以看出,残差随机分布在0值附近,总体离0值越近说明回归模型的拟合效果越好。对于偏离较大的残差点,可以认为是异常值点。
2、均方误差(Mean Squared Error, MSE)
均方误差是线性模型拟合过程中,最小化误差平方和(SSE)代价函数的平均值。MSE可以用于不同模型的比较,或是通过网格搜索进行参数调优,以及交叉验证等。
# 均方误差评价指标
from sklearn.metrics import mean_squared_error
print('MSE train: %.3f, test: %.3f' % (
mean_squared_error(y_train, y_train_pred,
mean_squared_error(y_test, y_test_pred))
))
结果: MSE train: 20.217, test: 28.147
3、决定系数
可以看做是MSE的标准化版本,用于更好地解释模型的性能。换句话说,决定系数是模型捕获相应反差的分数。
# 决定系数评价指标
from sklearn.metrics import r2_score
print('R^2 train: %.3f, test: %.3f' %
(r2_score(y_train, y_train_pred),
r2_score(y_test, y_test_pred)))
结果: R^2 train: 0.761, test: 0.662
整体代码
import matplotlib.pyplot as plt
import seaborn as sns
import pandas as pd
import numpy as np
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
# 数据读取
df = pd.read_csv('dataset/boston.csv', sep=',')
df.columns = ['CRIM', 'ZN', 'INDUS', 'CHAS',
'NOX', 'RM', 'AGE', 'DIS', 'RAD',
'TAX', 'PTRATIO', 'LSTAT', 'MEDV']
# 所有属性拟合线性模型
X = df.iloc[:, :-1].values
y = df[['MEDV']].values
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.3, random_state=0)
# 模型训练
slr = LinearRegression()
slr.fit(X_train, y_train)
y_train_pred = slr.predict(X_train)
y_test_pred = slr.predict(X_test)
# 残差评估方法
plt.scatter(y_train_pred, y_train_pred-y_train,
c='blue', marker='o', label='Training data')
plt.scatter(y_test_pred, y_test_pred-y_test,
c='lightgreen', marker='s', label='Test data')
plt.xlabel('Predicted values')
plt.ylabel('Residuals')
plt.legend(loc='upper left')
plt.hlines(y=0, xmin=-10, xmax=50, lw=2, colors='red')
plt.xlim([-10, 50])
plt.savefig('result/residuals_metric.png')
plt.show()
# 均方误差评价指标
from sklearn.metrics import mean_squared_error
print('MSE train: %.3f, test: %.3f' % (
mean_squared_error(y_train, y_train_pred),
mean_squared_error(y_test, y_test_pred))
)
# 决定系数评价指标
from sklearn.metrics import r2_score
print('R^2 train: %.3f, test: %.3f' %
(r2_score(y_train, y_train_pred),
r2_score(y_test, y_test_pred)))