由于网上有很多关于这个的介绍了,就简单记一下数据处理的过程。
任务上写着只把CRIM,RM,LSTAT作为特征,MEDV作为结果来进行预测,那么首先载入数据集
#Load the Boston Housing Data Set from sklearn.datasets and print it
from sklearn.datasets import load_boston
boston = load_boston()
print(boston)
然后把三列特征摘出来
df = pd.DataFrame(boston.data, columns = boston.feature_names)
df_x = df[['CRIM', 'RM','LSTAT']]
df_y = pd.DataFrame(boston.target)
print(df_x)
可以看到,特征数据是这样分布的:
CRIM RM LSTAT
0 0.00632 6.575 4.98
1 0.02731 6.421 9.14
2 0.02729 7.185 4.03
3 0.03237 6.998 2.94
4 0.06905 7.147 5.33
.. ... ... ...
501 0.06263 6.593 9.67
502 0.04527 6.120 9.08
503 0.06076 6.976 5.64
504 0.10959 6.794 6.48
505 0.04741 6.030 7.88
然后我们按照要求把训练及测试集合73分
#Initialize the linear regression model
reg = linear_model.LinearRegression()
#Split the data into 70% training and 30% testing data
#NOTE: We have to split the dependent variables (x) and the target or independent variable (y)
x_train, x_test, y_train, y_test = train_test_split(df_x, df_y, test_size=0.3, random_state=42)
#Train our model with the training data
reg.fit(x_train, y_train)
最后在预测指标的时候遇到了一些问题,就是pred和test数据怎么都报错
先看一下怎么算的:
error = []
#print(y_pred.shape)
pre = y_pred[:,0]
test = y_test.values
test = test[:,0]
p = pre.tolist()
t = test.tolist()
for i in range(len(y_pred)):
error.append(t[i] - p[i])
看一下pred的数据分布:
(152, 1)
[[25.95010416]
[31.04855815]
[18.2497264 ]
[26.3019832 ]
[19.72042823]
......
[16.17338714]
[40.34645114]
[21.0067563 ]
[19.39310502]]
可以看到这是152行1列的数据,所以我们把第一列的数据全部拿出来计算就行,因此pre = y_pred[:,0]
。至于test,如果按照同样的处理方式会发现报错,
因此先转换成ndarray,然后再取第一列。
之后进行for循环的时候,发现还是报错,在用这样的MAE计算方式时,需要将ndarray转换成list,再加一个数据转换tolist(),然后就可以了。
下面就是输出MAE和MSE
squaredError = []
absError = []
for val in error:
squaredError.append(val * val)#target-prediction之差平方
absError.append(abs(val))#误差绝对值
#print("Square Error: ", squaredError)
#print("Absolute Value of Error: ", absError)
print("MAE = ", sum(absError) / len(absError))#平均绝对误差MAE
print("MSE = ", sum(squaredError) / len(squaredError))#均方误差MSE
全部代码贴到下面:
import pandas as pd
import numpy as np
from sklearn import linear_model
from sklearn.model_selection import train_test_split
#Load the Boston Housing Data Set from sklearn.datasets and print it
from sklearn.datasets import load_boston
boston = load_boston()
print(boston)
#Transform the data set into a data frame
#NOTE: boston.data = the data we want,
# boston.feature_names = the column names of the data
# boston.target = Our target variable or the price of the houses
df = pd.DataFrame(boston.data, columns = boston.feature_names)
df_x = df[['CRIM', 'RM','LSTAT']]
df_y = pd.DataFrame(boston.target)
print(df_x)
#Initialize the linear regression model
reg = linear_model.LinearRegression()
#Split the data into 70% training and 30% testing data
#NOTE: We have to split the dependent variables (x) and the target or independent variable (y)
x_train, x_test, y_train, y_test = train_test_split(df_x, df_y, test_size=0.3, random_state=42)
#Train our model with the training data
reg.fit(x_train, y_train)
#print our price predictions on our test data
y_pred = reg.predict(x_test)
mse = np.sum((y_test - y_pred) ** 2) / len(y_test)
mae = np.sum(np.absolute(y_test - y_pred)) / len(y_test)
print(mae,mae)
error = []
print(y_pred.shape)
print(y_pred)
pre = y_pred[:,0]
test = y_test.values
test = test[:,0]
p = pre.tolist()
t = test.tolist()
for i in range(len(y_pred)):
error.append(t[i] - p[i])
squaredError = []
absError = []
for val in error:
squaredError.append(val * val)#target-prediction之差平方
absError.append(abs(val))#误差绝对值
#print("Square Error: ", squaredError)
#print("Absolute Value of Error: ", absError)
print("MAE = ", sum(absError) / len(absError))#平均绝对误差MAE
print("MSE = ", sum(squaredError) / len(squaredError))#均方误差MSE
#MAE = 4.111995393754926
#MSE = 29.975964330767486