箱型图介绍看链接:
一下Python是使用箱型图找到异常值并用线性插值把剔除异常后的缺失填充上。
核心代码
import pandas as pd
def wash_data(df: pd.DataFrame):
"""箱型图法"""
q1 = df.quantile(0.25)
q3 = df.quantile(0.75)
iqr = q3 - q1
mi = q1 - 1.5 * iqr
ma = q3 + 1.5 * iqr
# error = df[(df < mi) | (df > ma)]
data_c = df[(df >= mi) & (df <= ma)]
return data_c.interpolate().bfill() # 线性插值
测试代码
import pandas as pd
import numpy as np
if __name__ == '__main__':
df = pd.DataFrame(data=np.random.rand(10, 2))
df[0][1] = 100
df[0][2] = 99
print(df)
washedDf = wash_data(df)
print(washedDf)
输出结果
0 1
0 0.385158 0.469756
1 100.000000 0.090029
2 99.000000 0.807008
3 0.832849 0.672141
4 0.965382 0.847540
5 0.275084 0.003218
6 0.366026 0.211611
7 0.462773 0.226748
8 0.244522 0.916068
9 0.906602 0.029235
0 1
0 0.385158 0.469756
1 0.534389 0.090029
2 0.683619 0.807008
3 0.832849 0.672141
4 0.965382 0.847540
5 0.275084 0.003218
6 0.366026 0.211611
7 0.462773 0.226748
8 0.244522 0.916068
9 0.906602 0.029235
进程已结束,退出代码为 0