回归森林填补缺失值

用随机森林回归填补缺失值

在sklearn中，我们可以使用sklearn.impute.SimpleImputer来轻松地将均 值，中值，或者其他常用的数值填补到数据中，

在这个案例中，我们将使用均值，0，和随机森林回归来填补缺 失值，并验证四种状况下的拟合状况，

找出对使用的数据集来说佳的缺失值填补方法。

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.datasets import load_boston
from sklearn.impute import SimpleImputer
from sklearn.ensemble import RandomForestRegressor
from sklearn.model_selection import cross_val_score

dataset=load_boston()

dataset.data.shape

(506, 13)

• 共有506*13=6578数据

x_full,y_full=dataset.data,dataset.target
n_sample=x_full.shape[0]
n_features=x_full.shape[1]

n_sample

506

n_features

13

为完整数据放入缺失值

• 首先确定我们希望放入的缺失数据的比例，在这里我们假设是50%，那总共就要有3289个数据缺失

rng=np.random.RandomState(0)
missing_rate=0.5
n_missing_sample=int(np.float(n_sample*n_features*missing_rate))
# np.float向下取整

n_missing_sample

3289

所有数据要随机遍布在数据集的各行各列当中，
而一个缺失的数据会需要一个行索引和一个列索引
如果能够创造一个数组，包含3289个分0-506中间的行索引，和3289个分布在0~13之间的列索引，
那我们就可 以利用索引来为数据中的任意3289个位置赋空值
然后我们用0，均值和随机森林来填写这些缺失值，然后查看回归的结果如

missing_feature=rng.randint(0,n_features,n_missing_sample)
missing_sample=rng.randint(0,n_sample,n_missing_sample)

len(missing_feature)

3289

x_missing=x_full.copy()
y_missing=y_full.copy()
x_missing[missing_sample,missing_feature]=np.nan
x_missing=pd.DataFrame(x_missing)

x_missing.head()

	0	1	2	3	4	5	6	7	8	9	10	11	12
0	NaN	18.0	NaN	NaN	0.538	NaN	65.2	4.0900	1.0	296.0	NaN	NaN	4.98
1	0.02731	0.0	NaN	0.0	0.469	NaN	78.9	4.9671	2.0	NaN	NaN	396.9	9.14
2	0.02729	NaN	7.07	0.0	NaN	7.185	61.1	NaN	2.0	242.0	NaN	NaN	NaN
3	NaN	NaN	NaN	0.0	0.458	NaN	45.8	NaN	NaN	222.0	18.7	NaN	NaN
4	NaN	0.0	2.18	0.0	NaN	7.147	NaN	NaN	NaN	NaN	18.7	NaN	5.33

使用0和均值填补缺失值

• 使用均值

from sklearn.impute import SimpleImputer

imp_mean=SimpleImputer(missing_values=np.nan,strategy='mean')
x_missing_mean=imp_mean.fit_transform(x_missing)

• 使用0填充

imp_0=SimpleImputer(missing_values=np.nan,strategy="constant",fill_value=0)
x_missing_0=imp_0.fit_transform(x_missing) 
x_missing_reg=x_missing.copy()
sortindex=np.argsort(x_missing_reg.isnull().sum()).values

• 使用随机森林填补缺失值

# 分步测试
x_missing_reg=x_missing.copy()
sortindex=np.argsort(x_missing_reg.isnull().sum()).values
x_missing=pd.DataFrame(x_missing)

sortindex

df=x_missing

fillc=df.iloc[:,6]
fillc[:5]

df=pd.concat([df.iloc[:,df.columns !=6],pd.DataFrame(y_full)],axis=1)


df.head()

df_0=SimpleImputer(missing_values=np.nan,fill_value=0).fit_transform(df)

ytrain=fillc[fillc.notnull()]
ytest=fillc[fillc.isnull()]



xtrain=df_0[ytrain.index,:]
xtest = df_0[ytest.index,:] 

rfc = RandomForestRegressor(n_estimators=100)
rfc = rfc.fit(xtrain, ytrain)
Ypredict = rfc.predict(xtest)
#将填补好的特征返回到我的原始的特征矩阵中    
x_missing_reg.loc[x_missing_reg.iloc[:,6].isnull(),6] = Ypredict

for i in sortindex:
    df=x_missing_reg
    fillc=df.iloc[:,i]
    df=pd.concat([df.iloc[:,df.columns !=i],pd.DataFrame(y_full)],axis=1)
    df_0=SimpleImputer(missing_values=np.nan,fill_value=0).fit_transform(df)
    ytrain=fillc[fillc.notnull()]
    ytest=fillc[fillc.isnull()]
    xtrain=df_0[ytrain.index,: ]
    xtest=df_0[ytest.index,: ]
    rfc = RandomForestRegressor(n_estimators=100)
    rfc = rfc.fit(xtrain, ytrain) 
    ypredict=rfc.predict(xtest)
    x_missing_reg.loc[x_missing_reg.iloc[:,i].isnull(),i]=ypredict

x_missing_reg.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 506 entries, 0 to 505
Data columns (total 13 columns):
0     506 non-null float64
1     506 non-null float64
2     506 non-null float64
3     506 non-null float64
4     506 non-null float64
5     506 non-null float64
6     506 non-null float64
7     506 non-null float64
8     506 non-null float64
9     506 non-null float64
10    506 non-null float64
11    506 non-null float64
12    506 non-null float64
dtypes: float64(13)
memory usage: 51.5 KB

#对所有数据建模
X=[x_full,x_missing_mean,x_missing_0,x_missing_reg]
mse=[]
std=[]
for x in X:
    estimator=RandomForestRegressor(random_state=0,n_estimators=100)
    score=cross_val_score(estimator,x,y_full,scoring='neg_mean_squared_error',cv=5).mean()
    
    print(f"score is {score}")
    mse.append(score*-1)

score is -21.62860460743544
score is -40.84405476955929
score is -49.50657028893417
score is -18.62900872679091

x_labels = ['Full data','Zero Imputation','Mean Imputation','Regressor Imputation']
colors = ['r', 'g', 'b', 'orange']
plt.figure(figsize=(12, 6))
ax = plt.subplot(111) 
for i in np.arange(len(mse)):
     ax.barh(i, mse[i],color=colors[i], alpha=0.6, align='center') 
ax.set_title('Imputation Techniques with Boston Data') 
ax.set_xlim(left=np.min(mse) * 0.9,right=np.max(mse) * 1.1) 
ax.set_yticks(np.arange(len(mse))) 
ax.set_xlabel('MSE')
ax.set_yticklabels(x_labels) 
plt.show()

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-Di5mtKmG-1597149303962)(output_26_0.png)]

21.62860460743544

• 菜菜的sklearn学习得到 https://live.bilibili.com/12582510