1.概念和分类
模型融合用于对多个模型进行融合以得到优于单个模型的效果。
模型融合的方法主要有:
(1)加权求和融合
回归:算术平均融合,几何平均融合
分类:投票融合
综合:排序融合,log融合
(2) stacking/blending
将多个模型得到的预测结果concate之后再次进行模型预测
(3) boosting/bagging决策树方法
2. 代码实例
2.1 加权融合
def Weighted_method(test_pre1,test_pre2,w=[1/2,1/2]):
Weighted_result = w[0]*pd.Series(test_pre1)+w[1]*pd.Series(test_pre2)
return Weighted_result
## Init the Weight
w = [0.5,0.5]
## 测试验证集准确度
valA = Weighted_method(val_lgb,val_xgb,w)
print('MAE of weighed:',mean_absolute_error(y_val,valA))
2.2. Starking融合
## Starking
# 第一层
train_lgb_pred = model_lgb.predict(x_train)
train_xgb_pred = model_xgb.predict(x_train)
Strak_X_train = pd.DataFrame()
Strak_X_train['Method_1'] = train_lgb_pred
Strak_X_train['Method_2'] = train_xgb_pred
Strak_X_val = pd.DataFrame()
Strak_X_val['Method_1'] = val_lgb
Strak_X_val['Method_2'] = val_xgb
Strak_X_test = pd.DataFrame()
Strak_X_test['Method_1'] = subA_lgb
Strak_X_test['Method_2'] = subA_xgb
def build_model_lr(x_train,y_train):
reg_model = linear_model.LinearRegression()
reg_model.fit(x_train,y_train)
return reg_model
## level2-method
model_lr_Stacking = build_model_lr(Strak_X_train,y_train)
## 验证集
val_pre_Stacking = model_lr_Stacking.predict(Strak_X_val)
print('MAE of Stacking-LR:',mean_absolute_error(y_val,val_pre_Stacking))
结果:
在验证集下的结果是用加权求和的方法更好。