这里采用三种模型进行拟合预测对比,分别是线性分位数回归,XGB结合分位数,LightGBM结合分位数:
alphas = np.arange(5, 100, 5) / 100.0
print(alphas)
mse_qr, mse_xgb, mse_lgb = [], [], []
r2_qr, r2_xgb, r2_lgb = [], [], []
qr_pred,xgb_pred,lgb_pred={},{},{}
# Train and evaluate
for alpha in alphas:
# Quantile Regression
model_qr = QuantReg(y_train, sm.add_constant(X_train)).fit(q=alpha)
model_pred=model_qr.predict(sm.add_constant(X_test))
mse_qr.append(mean_squared_error(y_test,model_pred ))
r2_qr.append(r2_score(y_test,model_pred))
# XGBoost
model_xgb = xgb.train({"objective": "reg:quantileerror", 'quantile_alpha': alpha},
xgb.QuantileDMatrix(X_train, y_train), num_boost_round=100)
model_pred=model_xgb.predict(xgb.DMatrix(X_test))
mse_xgb.append(mean_squared_error(y_test,model_pred ))
r2_xgb.append(r2_score(y_test,model_pred))
# LightGBM
model_lgb = lgb.train({'objective': 'quantile', 'alpha': alpha,'force_col_wise': True,},
lgb.Dataset(X_train, y_train), num_boost_round=100)
model_pred=model_lgb.predict(X_test)
mse_lgb.append(mean_squared_error(y_test,model_pred))
r2_lgb.append(r2_score(y_test,model_pred))
if alpha in [0.1,0.5,0.9]:
qr_pred[alpha]=model_qr.predict(sm.add_constant(X_test))
xgb_pred[alpha]=model_xgb.predict(xgb.DMatrix(X_test))
lgb_pred[alpha]=model_lgb.predict(X_test)
分位点为0.1,0.5,0.9时记录一下,方便画图查看。
然后画出三种模型在不同分位点下的误差和拟合优度对比:
plt.figure(figsize=(7, 5),