Python机器学习18——Xgboost和Lightgbm结合分位数回归(机器学习与传统统计学结合)_xgboost做分位数回归

这里采用三种模型进行拟合预测对比，分别是线性分位数回归，XGB结合分位数，LightGBM结合分位数：

alphas = np.arange(5, 100, 5) / 100.0
print(alphas)
mse_qr, mse_xgb, mse_lgb = [], [], []
r2_qr, r2_xgb, r2_lgb = [], [], []
qr_pred,xgb_pred,lgb_pred={},{},{}

# Train and evaluate
for alpha in alphas:
    # Quantile Regression
    model_qr = QuantReg(y_train, sm.add_constant(X_train)).fit(q=alpha)
    model_pred=model_qr.predict(sm.add_constant(X_test))
    mse_qr.append(mean_squared_error(y_test,model_pred ))
    r2_qr.append(r2_score(y_test,model_pred))
    
    # XGBoost
    model_xgb = xgb.train({"objective": "reg:quantileerror", 'quantile_alpha': alpha}, 
                          xgb.QuantileDMatrix(X_train, y_train), num_boost_round=100)
    model_pred=model_xgb.predict(xgb.DMatrix(X_test))
    mse_xgb.append(mean_squared_error(y_test,model_pred ))
    r2_xgb.append(r2_score(y_test,model_pred))
    
    # LightGBM
    model_lgb = lgb.train({'objective': 'quantile', 'alpha': alpha,'force_col_wise': True,}, 
                          lgb.Dataset(X_train, y_train), num_boost_round=100)
    
    model_pred=model_lgb.predict(X_test)
    mse_lgb.append(mean_squared_error(y_test,model_pred))
    r2_lgb.append(r2_score(y_test,model_pred))
    
    if alpha in [0.1,0.5,0.9]:
        qr_pred[alpha]=model_qr.predict(sm.add_constant(X_test))
        xgb_pred[alpha]=model_xgb.predict(xgb.DMatrix(X_test))
        lgb_pred[alpha]=model_lgb.predict(X_test)

分位点为0.1,0.5,0.9时记录一下，方便画图查看。

然后画出三种模型在不同分位点下的误差和拟合优度对比：

plt.figure(figsize=(7, 5),