sklearn 缺失值处理器： SimpleImputer

class sklearn.impute.SimpleImputer(*, missing_values=nan, strategy=‘mean’, fill_value=None, verbose=‘deprecated’, copy=True, add_indicator=False)[source]
New in version 0.20: SimpleImputer replaces the previous sklearn.preprocessing.Imputer estimator which is now removed.

注意：

Imputer 只接受DataFrame类型
Dataframe 中必须全部为数值属性

strategy全是按列处理，与原来的Imputer相比，取消了参数axis=0,1。

demo.py 如下：

参考链接: 缺失值处理器： Imputer

import numpy as np
import pandas as pd
from sklearn.impute import SimpleImputer

df=pd.DataFrame([["XXL", 8, "black", "class 1", 22],
["L", np.nan, "gray", "class 2", 20],
["XL", 10, "blue", "class 2", 19],
["M", np.nan, "orange", "class 1", 17],
["M", 11, "green", "class 3", np.nan],
["M", 7, "red", "class 1", 22]])

df.columns=["size", "price", "color", "class", "boh"]
print(df)
# 输出：
'''
  size  price   color    class   boh
0  XXL    8.0   black  class 1  22.0
1    L    NaN    gray  class 2  20.0
2   XL   10.0    blue  class 2  19.0
3    M    NaN  orange  class 1  17.0
4    M   11.0   green  class 3   NaN
5    M    7.0     red  class 1  22.0
'''

im = SimpleImputer(missing_values=np.nan,strategy='median')
# 先只处理price列的数据， 注意使用的是   df[['price']]   这样返回的是一个DataFrame类型的数据！
# df['price']返回的是Series类型，不能放入SimpleImputer
df1 = im.fit_transform(df[['price']])
print(df1)
# 输出一个二维数组：
'''
[[ 8.]
 [ 9.]
 [10.]
 [ 9.]
 [11.]
 [ 7.]]
 '''
df['price']=df1 # 替换原数据
print(df)
# 输出：
'''
  size  price   color    class   boh
0  XXL    8.0   black  class 1  22.0
1    L    9.0    gray  class 2  20.0
2   XL   10.0    blue  class 2  19.0
3    M    9.0  orange  class 1  17.0
4    M   11.0   green  class 3   NaN
5    M    7.0     red  class 1  22.0
'''

—————————————————————————————————————————————

2022.7.29 Kaggle_titanic_competition补充

缺失值处理

数值类型：用平均值/中位数取代

分类数据：用常见类别取代 (most_frequent关键字)，如果缺失值较多，用U(Unknown)取代(constant,fill_value)

使用模型预测缺失值：例如 KNN (我还没用过)

————————————————————————————————————————————

// SimpleImputer也可以一次性传入多列的 DataFrame

// An highlighted block
# sklearn.imputer处理缺失值--数值类型
im = SimpleImputer(missing_values=np.nan,strategy='median')
df1 = im.fit_transform(data[['Age','Fare']])
data[['Age','Fare']] = df1

或者就简使用 .fillna()

data['Age'] = data['Age'].fillna(data['Age'].median())

SimpleImputer也可以处理字符串类型的DataFrame缺失值

// strategy---str, default=’mean’

If “most_frequent”, then replace missing using the most frequent value along each column. Can be used with strings or numeric data. If there is more than one such value, only the smallest is returned.

If “constant”, then replace missing values with fill_value. Can be used with strings or numeric data.
（当strategy='constant'时，用fill_value=''来填充所有 np.nan 缺失值）

data['Embarked'].value_counts()查看港口的分类数据情况

Out[ ]:
S    914
C    270
Q    123
Name: Embarked, dtype: int64

//发现S港口最多，把缺失值填充为S港口
data['Embarked'] = data['Embarked'].fillna('S')
//也可以使用	SimpleImputer(missing_values=np.nan,strategy='most_frequent')

// 由于特征Cabin船舱号缺失数据较多，根据特征工程思路：填充为U，表示未知
im1 = SimpleImputer(missing_values=np.nan,strategy='constant',fill_value='U')
data[['Embarked']] = im1.fit_transform(data[['Embarked']] )