机器学习实战 —— 工业蒸汽量预测（六）

ZShiJ

已于 2024-05-07 18:00:56 修改

阅读量1.2k

点赞数 43

分类专栏：机器学习文章标签：机器学习人工智能

于 2024-05-05 07:00:00 首次发布

本文链接：https://blog.csdn.net/m0_53054984/article/details/138225809

版权

机器学习专栏收录该内容

20 篇文章 2 订阅

订阅专栏

🌟欢迎来到 我的博客 —— 探索技术的无限可能！

🌟博客的简介（文章目录）

文章描述

数据分析：查看变量间相关性以及找出关键变量。
机器学习实战 —— 工业蒸汽量预测（一）
数据特征工程对数据精进：异常值处理、归一化处理以及特征降维。
机器学习实战 —— 工业蒸汽量预测（二）
模型训练(涉及主流ML模型)：决策树、随机森林，lightgbm等。
机器学习实战 —— 工业蒸汽量预测（三）
模型验证：评估指标以及交叉验证等。
机器学习实战 —— 工业蒸汽量预测（四）
特征优化：用lgb对特征进行优化。
机器学习实战 —— 工业蒸汽量预测（五）
模型融合：进行基于stacking方式模型融合。
机器学习实战 —— 工业蒸汽量预测（六）

背景描述

背景介绍

火力发电的基本原理是：燃料在燃烧时加热水生成蒸汽，蒸汽压力推动汽轮机旋转，然后汽轮机带动发电机旋转，产生电能。在这一系列的能量转化中，影响发电效率的核心是锅炉的燃烧效率，即燃料燃烧加热水产生高温高压蒸汽。锅炉的燃烧效率的影响因素很多，包括锅炉的可调参数，如燃烧给量，一二次风，引风，返料风，给水水量；以及锅炉的工况，比如锅炉床温、床压，炉膛温度、压力，过热器的温度等。

相关描述

经脱敏后的锅炉传感器采集的数据（采集频率是分钟级别），根据锅炉的工况，预测产生的蒸汽量。

结果评估

预测结果以mean square error作为评判标准。

数据说明

数据分成训练数据（train.txt）和测试数据（test.txt），其中字段”V0”-“V37”，这38个字段是作为特征变量，”target”作为目标变量。选手利用训练数据训练出模型，预测测试数据的目标变量，排名结果依据预测结果的MSE（mean square error）。

数据来源

http://tianchi-media.oss-cn-beijing.aliyuncs.com/DSW/Industrial_Steam_Forecast/zhengqi_test.txt

http://tianchi-media.oss-cn-beijing.aliyuncs.com/DSW/Industrial_Steam_Forecast/zhengqi_train.txt

实战内容

6.模型融合

下面把上篇关键流程在跑一边

导入包和数据

import warnings
warnings.filterwarnings("ignore")
import matplotlib.pyplot as plt
plt.rcParams.update({'figure.max_open_warning': 0})
import seaborn as sns
import pandas as pd
import numpy as np
from scipy import stats
from sklearn.model_selection import train_test_split
from sklearn.model_selection import GridSearchCV, RepeatedKFold, cross_val_score,cross_val_predict,KFold
from sklearn.metrics import make_scorer,mean_squared_error
from sklearn.linear_model import LinearRegression, Lasso, Ridge, ElasticNet
from sklearn.svm import LinearSVR, SVR
from sklearn.neighbors import KNeighborsRegressor
from sklearn.ensemble import RandomForestRegressor, GradientBoostingRegressor,AdaBoostRegressor
from xgboost import XGBRegressor
from sklearn.preprocessing import PolynomialFeatures,MinMaxScaler,StandardScaler

with open("./zhengqi_train.txt")  as fr:
    data_train=pd.read_table(fr,sep="\t")
with open("./zhengqi_test.txt") as fr_test:
    data_test=pd.read_table(fr_test,sep="\t")

合并数据

data_train["oringin"]="train"
data_test["oringin"]="test"
data_all=pd.concat([data_train,data_test],axis=0,ignore_index=True)

删除相关特征

data_all.drop(["V5","V9","V11","V17","V22","V28"],axis=1,inplace=True)

数据最大最小归一化

cols_numeric=list(data_all.columns)
cols_numeric.remove("oringin")
def scale_minmax(col):
    return (col-col.min())/(col.max()-col.min())
scale_cols = [col for col in cols_numeric if col!='target']
data_all[scale_cols] = data_all[scale_cols].apply(scale_minmax,axis=0)

画图：探查特征和标签相关信息

在这里插入图片描述

对特征进行Box-Cox变换，使其满足正态性

Box-Cox变换是Box和Cox在1964年提出的一种广义幂变换方法，是统计建模中常用的一种数据变换，用于连续的响应变量不满足正态分布的情况。Box-Cox变换之后，可以一定程度上减小不可观测的误差和预测变量的相关性。Box-Cox变换的主要特点是引入一个参数，通过数据本身估计该参数进而确定应采取的数据变换形式，Box-Cox变换可以明显地改善数据的正态性、对称性和方差相等性，对许多实际数据都是行之有效的

cols_transform=data_all.columns[0:-2]
for col in cols_transform:   
    # transform column
    data_all.loc[:,col], _ = stats.boxcox(data_all.loc[:,col]+1)

标签数据统计转换后的数据，计算分位数画图展示（基于正态分布）

print(data_all.target.describe())

plt.figure(figsize=(12,4))
plt.subplot(1,2,1)
sns.distplot(data_all.target.dropna() , fit=stats.norm);
plt.subplot(1,2,2)
_=stats.probplot(data_all.target.dropna(), plot=plt)

在这里插入图片描述

标签数据对数变换数据，使数据更符合正态，并画图展示

sp = data_train.target
data_train.target1 =np.power(1.5,sp)
print(data_train.target1.describe())

plt.figure(figsize=(12,4))
plt.subplot(1,2,1)
sns.distplot(data_train.target1.dropna(),fit=stats.norm);
plt.subplot(1,2,2)
_=stats.probplot(data_train.target1.dropna(), plot=plt)

在这里插入图片描述

获取训练和测试数据

def get_training_data():

    from sklearn.model_selection import train_test_split
    df_train = data_all[data_all["oringin"]=="train"]
    df_train["label"]=data_train.target1

    y = df_train.target
    X = df_train.drop(["oringin","target","label"],axis=1)
    X_train,X_valid,y_train,y_valid=train_test_split(X,y,test_size=0.3,random_state=100)
    return X_train,X_valid,y_train,y_valid

def get_test_data():
    df_test = data_all[data_all["oringin"]=="test"].reset_index(drop=True)
    return df_test.drop(["oringin","target"],axis=1)

评分函数

from sklearn.metrics import make_scorer

def rmse(y_true, y_pred):
    diff = y_pred - y_true
    sum_sq = sum(diff**2)    
    n = len(y_pred)   
    
    return np.sqrt(sum_sq/n)
def mse(y_ture,y_pred):
    return mean_squared_error(y_ture,y_pred)

rmse_scorer = make_scorer(rmse, greater_is_better=False)
mse_scorer = make_scorer(mse, greater_is_better=False)

获取异常数据，并画图

def find_outliers(model, X, y, sigma=3):
    try:
        y_pred = pd.Series(model.predict(X), index=y.index)
    except:
        model.fit(X,y)
        y_pred = pd.Series(model.predict(X), index=y.index)
        
    resid = y - y_pred
    mean_resid = resid.mean()
    std_resid = resid.std()

    z = (resid - mean_resid)/std_resid    
    outliers = z[abs(z)>sigma].index
    
    print('R2=',model.score(X,y))
    print('rmse=',rmse(y, y_pred))
    print("mse=",mean_squared_error(y,y_pred))
    print('---------------------------------------')

    print('mean of residuals:',mean_resid)
    print('std of residuals:',std_resid)
    print('---------------------------------------')

    print(len(outliers),'outliers:')
    print(outliers.tolist())

    plt.figure(figsize=(15,5))
    ax_131 = plt.subplot(1,3,1)
    plt.plot(y,y_pred,'.')
    plt.plot(y.loc[outliers],y_pred.loc[outliers],'ro')
    plt.legend(['Accepted','Outlier'])
    plt.xlabel('y')
    plt.ylabel('y_pred');

    ax_132=plt.subplot(1,3,2)
    plt.plot(y,y-y_pred,'.')
    plt.plot(y.loc[outliers],y.loc[outliers]-y_pred.loc[outliers],'ro')
    plt.legend(['Accepted','Outlier'])
    plt.xlabel('y')
    plt.ylabel('y - y_pred');

    ax_133=plt.subplot(1,3,3)
    z.plot.hist(bins=50,ax=ax_133)
    z.loc[outliers].plot.hist(color='r',bins=50,ax=ax_133)
    plt.legend(['Accepted','Outlier'])
    plt.xlabel('z')
    
    plt.savefig('outliers.png')
    
    return outliers

from sklearn.linear_model import Ridge
X_train, X_valid,y_train,y_valid = get_training_data()
test=get_test_data()

outliers = find_outliers(Ridge(), X_train, y_train)

X_outliers=X_train.loc[outliers]
y_outliers=y_train.loc[outliers]
X_t=X_train.drop(outliers)
y_t=y_train.drop(outliers)

在这里插入图片描述

# 使用删除异常的数据进行模型训练
def get_trainning_data_omitoutliers():
    y1=y_t.copy()
    X1=X_t.copy()
    return X1,y1

# 采用网格搜索训练模型
from sklearn.preprocessing import StandardScaler
def train_model(model, param_grid=[], X=[], y=[], 
                splits=5, repeats=5):

    if len(y)==0:
        X,y = get_trainning_data_omitoutliers()
    
    rkfold = RepeatedKFold(n_splits=splits, n_repeats=repeats)
    
    if len(param_grid)>0:
        gsearch = GridSearchCV(model, param_grid, cv=rkfold,
                               scoring="neg_mean_squared_error",
                               verbose=1, return_train_score=True)

        gsearch.fit(X,y)

        model = gsearch.best_estimator_        
        best_idx = gsearch.best_index_

        grid_results = pd.DataFrame(gsearch.cv_results_)       
        cv_mean = abs(grid_results.loc[best_idx,'mean_test_score'])
        cv_std = grid_results.loc[best_idx,'std_test_score']
 
    else:
        grid_results = []
        cv_results = cross_val_score(model, X, y, scoring="neg_mean_squared_error", cv=rkfold)
        cv_mean = abs(np.mean(cv_results))
        cv_std = np.std(cv_results)
    
    cv_score = pd.Series({'mean':cv_mean,'std':cv_std})

    y_pred = model.predict(X)
      
    print('----------------------')
    print(model)
    print('----------------------')
    print('score=',model.score(X,y))
    print('rmse=',rmse(y, y_pred))
    print('mse=',mse(y, y_pred))
    print('cross_val: mean=',cv_mean,', std=',cv_std)

    y_pred = pd.Series(y_pred,index=y.index)
    resid = y - y_pred
    mean_resid = resid.mean()
    std_resid = resid.std()
    z = (resid - mean_resid)/std_resid    
    n_outliers = sum(abs(z)>3)
    
    plt.figure(figsize=(15,5))
    ax_131 = plt.subplot(1,3,1)
    plt.plot(y,y_pred,'.')
    plt.xlabel('y')
    plt.ylabel('y_pred');
    plt.title('corr = {:.3f}'.format(np.corrcoef(y,y_pred)[0][1]))
    ax_132=plt.subplot(1,3,2)
    plt.plot(y,y-y_pred,'.')
    plt.xlabel('y')
    plt.ylabel('y - y_pred');
    plt.title('std resid = {:.3f}'.format(std_resid))
    
    ax_133=plt.subplot(1,3,3)
    z.plot.hist(bins=50,ax=ax_133)
    plt.xlabel('z')
    plt.title('{:.0f} samples with z>3'.format(n_outliers))

    return model, cv_score, grid_results

opt_models = dict()
score_models = pd.DataFrame(columns=['mean','std'])

splits=5
repeats=5

6.1 单一模型预测效果

6.1.1 岭回归

model = 'Ridge'

opt_models[model] = Ridge()
alph_range = np.arange(0.25,6,0.25)
param_grid = {'alpha': alph_range}

opt_models[model],cv_score,grid_results = train_model(opt_models[model], param_grid=param_grid, 
                                              splits=splits, repeats=repeats)

cv_score.name = model
score_models = score_models.append(cv_score)

plt.figure()
plt.errorbar(alph_range, abs(grid_results['mean_test_score']),
             abs(grid_results['std_test_score'])/np.sqrt(splits*repeats))
plt.xlabel('alpha')
plt.ylabel('score')

在这里插入图片描述

6.1.2 Lasso回归

model = 'Lasso'

opt_models[model] = Lasso()
alph_range = np.arange(1e-4,1e-3,4e-5)
param_grid = {'alpha': alph_range}

opt_models[model], cv_score, grid_results = train_model(opt_models[model], param_grid=param_grid, 
                                              splits=splits, repeats=repeats)

cv_score.name = model
score_models = score_models.append(cv_score)

plt.figure()
plt.errorbar(alph_range, abs(grid_results['mean_test_score']),abs(grid_results['std_test_score'])/np.sqrt(splits*repeats))
plt.xlabel('alpha')
plt.ylabel('score')

在这里插入图片描述

6.1.3 ElasticNet 回归

model ='ElasticNet'
opt_models[model] = ElasticNet()

param_grid = {'alpha': np.arange(1e-4,1e-3,1e-4),
              'l1_ratio': np.arange(0.1,1.0,0.1),
              'max_iter':[100000]}

opt_models[model], cv_score, grid_results = train_model(opt_models[model], param_grid=param_grid, 
                                              splits=splits, repeats=1)

cv_score.name = model
score_models = score_models.append(cv_score)

在这里插入图片描述

6.1.4 SVR回归

model='LinearSVR'
opt_models[model] = LinearSVR()

crange = np.arange(0.1,1.0,0.1)
param_grid = {'C':crange,
             'max_iter':[1000]}

opt_models[model], cv_score, grid_results = train_model(opt_models[model], param_grid=param_grid, 
                                              splits=splits, repeats=repeats)


cv_score.name = model
score_models = score_models.append(cv_score)

plt.figure()
plt.errorbar(crange, abs(grid_results['mean_test_score']),abs(grid_results['std_test_score'])/np.sqrt(splits*repeats))
plt.xlabel('C')
plt.ylabel('score')

在这里插入图片描述

6.1.5 KNN最近邻

model = 'KNeighbors'
opt_models[model] = KNeighborsRegressor()

param_grid = {'n_neighbors':np.arange(3,11,1)}

opt_models[model], cv_score, grid_results = train_model(opt_models[model], param_grid=param_grid, 
                                              splits=splits, repeats=1)

cv_score.name = model
score_models = score_models.append(cv_score)

plt.figure()
plt.errorbar(np.arange(3,11,1), abs(grid_results['mean_test_score']),abs(grid_results['std_test_score'])/np.sqrt(splits*1))
plt.xlabel('n_neighbors')
plt.ylabel('score')

在这里插入图片描述

6.1.6 GBDT 模型

model = 'GradientBoosting'
opt_models[model] = GradientBoostingRegressor()

param_grid = {'n_estimators':[150,250,350],
              'max_depth':[1,2,3],
              'min_samples_split':[5,6,7]}

opt_models[model], cv_score, grid_results = train_model(opt_models[model], param_grid=param_grid, 
                                              splits=splits, repeats=1)

cv_score.name = model
score_models = score_models.append(cv_score)

在这里插入图片描述

6.1.7XGB模型

model = 'XGB'
opt_models[model] = XGBRegressor()

param_grid = {'n_estimators':[100,200,300,400,500],
              'max_depth':[1,2,3],
             }

opt_models[model], cv_score,grid_results = train_model(opt_models[model], param_grid=param_grid, 
                                              splits=splits, repeats=1)

cv_score.name = model
score_models = score_models.append(cv_score)

在这里插入图片描述

6.1.8 随机森林模型

model = 'RandomForest'
opt_models[model] = RandomForestRegressor()

param_grid = {'n_estimators':[100,150,200],
              'max_features':[8,12,16,20,24],
              'min_samples_split':[2,4,6]}

opt_models[model], cv_score, grid_results = train_model(opt_models[model], param_grid=param_grid, 
                                              splits=5, repeats=1)

cv_score.name = model
score_models = score_models.append(cv_score)

在这里插入图片描述

6.2 模型预测–多模型Bagging

def model_predict(test_data,test_y=[],stack=False):

    i=0
    y_predict_total=np.zeros((test_data.shape[0],))
    for model in opt_models.keys():
        if model!="LinearSVR" and model!="KNeighbors":
            y_predict=opt_models[model].predict(test_data)
            y_predict_total+=y_predict
            i+=1
        if len(test_y)>0:
            print("{}_mse:".format(model),mean_squared_error(y_predict,test_y))
    y_predict_mean=np.round(y_predict_total/i,3)
    if len(test_y)>0:
        print("mean_mse:",mean_squared_error(y_predict_mean,test_y))
    else:
        y_predict_mean=pd.Series(y_predict_mean)
        return y_predict_mean

# Bagging预测
model_predict(X_valid,y_valid)

在这里插入图片描述

6.3 模型融合Stacking

6.3.1 模型融合stacking简单示例

import numpy as np
import matplotlib.pyplot as plt
import matplotlib.gridspec as gridspec
import itertools
from sklearn.linear_model import LogisticRegression
from sklearn.svm import SVC
from sklearn.ensemble import RandomForestClassifier

##主要使用pip install mlxtend安装mlxtend
from mlxtend.classifier import EnsembleVoteClassifier
from mlxtend.data import iris_data
from mlxtend.plotting import plot_decision_regions
%matplotlib inline

clf1 = LogisticRegression(random_state=0)
clf2 = RandomForestClassifier(random_state=0)
clf3 = SVC(random_state=0, probability=True)
eclf = EnsembleVoteClassifier(clfs=[clf1, clf2, clf3], weights=[2, 1, 1], voting='soft')

X, y = iris_data()
X = X[:,[0, 2]]

gs = gridspec.GridSpec(2, 2)
fig = plt.figure(figsize=(10, 8))

for clf, lab, grd in zip([clf1, clf2, clf3, eclf],
                         ['Logistic Regression', 'Random Forest', 'RBF kernel SVM', 'Ensemble'],
                         itertools.product([0, 1], repeat=2)):
    clf.fit(X, y)
    ax = plt.subplot(gs[grd[0], grd[1]])
    fig = plot_decision_regions(X=X, y=y, clf=clf, legend=2)
    plt.title(lab)
plt.show()

在这里插入图片描述

6.3.2工业蒸汽多模型融合stacking

from sklearn.model_selection import train_test_split
import pandas as pd
import numpy as np
from scipy import sparse
import xgboost
import lightgbm

from sklearn.ensemble import RandomForestRegressor,AdaBoostRegressor,GradientBoostingRegressor,ExtraTreesRegressor
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error

def stacking_reg(clf,train_x,train_y,test_x,clf_name,kf,label_split=None):
    train=np.zeros((train_x.shape[0],1))
    test=np.zeros((test_x.shape[0],1))
    test_pre=np.empty((folds,test_x.shape[0],1))
    cv_scores=[]
    for i,(train_index,test_index) in enumerate(kf.split(train_x,label_split)):       
        tr_x=train_x[train_index]
        tr_y=train_y[train_index]
        te_x=train_x[test_index]
        te_y = train_y[test_index]
        if clf_name in ["rf","ada","gb","et","lr","lsvc","knn"]:
            clf.fit(tr_x,tr_y)
            pre=clf.predict(te_x).reshape(-1,1)
            train[test_index]=pre
            test_pre[i,:]=clf.predict(test_x).reshape(-1,1)
            cv_scores.append(mean_squared_error(te_y, pre))
        elif clf_name in ["xgb"]:
            train_matrix = clf.DMatrix(tr_x, label=tr_y, missing=-1)
            test_matrix = clf.DMatrix(te_x, label=te_y, missing=-1)
            z = clf.DMatrix(test_x, label=te_y, missing=-1)
            params = {'booster': 'gbtree',
                      'eval_metric': 'rmse',
                      'gamma': 1,
                      'min_child_weight': 1.5,
                      'max_depth': 5,
                      'lambda': 10,
                      'subsample': 0.7,
                      'colsample_bytree': 0.7,
                      'colsample_bylevel': 0.7,
                      'eta': 0.03,
                      'tree_method': 'exact',
                      'seed': 2017,
                      'nthread': 12
                      }
            num_round = 1000 #记得修改
            early_stopping_rounds = 10 #修改
            watchlist = [(train_matrix, 'train'),
                         (test_matrix, 'eval')
                         ]
            if test_matrix:
                model = clf.train(params, train_matrix, num_boost_round=num_round,evals=watchlist,
                                  early_stopping_rounds=early_stopping_rounds
                                  )
                pre= model.predict(test_matrix,ntree_limit=model.best_ntree_limit).reshape(-1,1)
                train[test_index]=pre
                test_pre[i, :]= model.predict(z, ntree_limit=model.best_ntree_limit).reshape(-1,1)
                cv_scores.append(mean_squared_error(te_y, pre))

        elif clf_name in ["lgb"]:
            train_matrix = clf.Dataset(tr_x, label=tr_y)
            test_matrix = clf.Dataset(te_x, label=te_y)
            params = {
                      'boosting_type': 'gbdt',
                      'objective': 'regression_l2',
                      'metric': 'mse',
                      'min_child_weight': 1.5,
                      'num_leaves': 2**5,
                      'lambda_l2': 10,
                      'subsample': 0.7,
                      'colsample_bytree': 0.7,
                      'colsample_bylevel': 0.7,
                      'learning_rate': 0.03,
                      'tree_method': 'exact',
                      'seed': 2017,
                      'nthread': 12,
                      'silent': True,
                      }
            num_round = 1000 #修改
            early_stopping_rounds = 10 #修改
            if test_matrix:
                model = clf.train(params, train_matrix,num_round,valid_sets=test_matrix,
                                  early_stopping_rounds=early_stopping_rounds
                                  )
                pre= model.predict(te_x,num_iteration=model.best_iteration).reshape(-1,1)
                train[test_index]=pre
                test_pre[i, :]= model.predict(test_x, num_iteration=model.best_iteration).reshape(-1,1)
                cv_scores.append(mean_squared_error(te_y, pre))
        else:
            raise IOError("Please add new clf.")
        print("%s now score is:"%clf_name,cv_scores)
    test[:]=test_pre.mean(axis=0)
    print("%s_score_list:"%clf_name,cv_scores)
    print("%s_score_mean:"%clf_name,np.mean(cv_scores))
    return train.reshape(-1,1),test.reshape(-1,1)

模型融合stacking基学习器

def rf_reg(x_train, y_train, x_valid, kf, label_split=None):
    randomforest = RandomForestRegressor(n_estimators=100, max_depth=20, n_jobs=-1, random_state=2017, max_features="auto",verbose=1)
    rf_train, rf_test = stacking_reg(randomforest, x_train, y_train, x_valid, "rf", kf, label_split=label_split)
    return rf_train, rf_test,"rf_reg"

def ada_reg(x_train, y_train, x_valid, kf, label_split=None):
    adaboost = AdaBoostRegressor(n_estimators=30, random_state=2017, learning_rate=0.01)
    ada_train, ada_test = stacking_reg(adaboost, x_train, y_train, x_valid, "ada", kf, label_split=label_split)
    return ada_train, ada_test,"ada_reg"

def gb_reg(x_train, y_train, x_valid, kf, label_split=None):
    gbdt = GradientBoostingRegressor(learning_rate=0.04, n_estimators=100, subsample=0.8, random_state=2017,max_depth=5,verbose=1)
    gbdt_train, gbdt_test = stacking_reg(gbdt, x_train, y_train, x_valid, "gb", kf, label_split=label_split)
    return gbdt_train, gbdt_test,"gb_reg"

def et_reg(x_train, y_train, x_valid, kf, label_split=None):
    extratree = ExtraTreesRegressor(n_estimators=100, max_depth=35, max_features="auto", n_jobs=-1, random_state=2017,verbose=1)
    et_train, et_test = stacking_reg(extratree, x_train, y_train, x_valid, "et", kf, label_split=label_split)
    return et_train, et_test,"et_reg"

def lr_reg(x_train, y_train, x_valid, kf, label_split=None):
    lr_reg=LinearRegression(n_jobs=-1)
    lr_train, lr_test = stacking_reg(lr_reg, x_train, y_train, x_valid, "lr", kf, label_split=label_split)
    return lr_train, lr_test, "lr_reg"

def xgb_reg(x_train, y_train, x_valid, kf, label_split=None):
    xgb_train, xgb_test = stacking_reg(xgboost, x_train, y_train, x_valid, "xgb", kf, label_split=label_split)
    return xgb_train, xgb_test,"xgb_reg"

def lgb_reg(x_train, y_train, x_valid, kf, label_split=None):
    lgb_train, lgb_test = stacking_reg(lightgbm, x_train, y_train, x_valid, "lgb", kf, label_split=label_split)
    return lgb_train, lgb_test,"lgb_reg"

模型融合stacking预测

def stacking_pred(x_train, y_train, x_valid, kf, clf_list, label_split=None, clf_fin="lgb", if_concat_origin=True):
    for k, clf_list in enumerate(clf_list):
        clf_list = [clf_list]
        column_list = []
        train_data_list=[]
        test_data_list=[]
        for clf in clf_list:
            train_data,test_data,clf_name=clf(x_train, y_train, x_valid, kf, label_split=label_split)
            train_data_list.append(train_data)
            test_data_list.append(test_data)
            column_list.append("clf_%s" % (clf_name))
    train = np.concatenate(train_data_list, axis=1)
    test = np.concatenate(test_data_list, axis=1)
    
    if if_concat_origin:
        train = np.concatenate([x_train, train], axis=1)
        test = np.concatenate([x_valid, test], axis=1)
    print(x_train.shape)
    print(train.shape)
    print(clf_name)
    print(clf_name in ["lgb"])
    if clf_fin in ["rf","ada","gb","et","lr","lsvc","knn"]:
        if clf_fin in ["rf"]:
            clf = RandomForestRegressor(n_estimators=100, max_depth=20, n_jobs=-1, random_state=2017, max_features="auto",verbose=1)
        elif clf_fin in ["ada"]:
            clf = AdaBoostRegressor(n_estimators=30, random_state=2017, learning_rate=0.01)
        elif clf_fin in ["gb"]:
            clf = GradientBoostingRegressor(learning_rate=0.04, n_estimators=100, subsample=0.8, random_state=2017,max_depth=5,verbose=1)
        elif clf_fin in ["et"]:
            clf = ExtraTreesRegressor(n_estimators=100, max_depth=35, max_features="auto", n_jobs=-1, random_state=2017,verbose=1)
        elif clf_fin in ["lr"]:
            clf = LinearRegression(n_jobs=-1)
        clf.fit(train, y_train)
        pre = clf.predict(test).reshape(-1,1)
        return pred
    elif clf_fin in ["xgb"]:
        clf = xgboost
        train_matrix = clf.DMatrix(train, label=y_train, missing=-1)
        test_matrix = clf.DMatrix(train, label=y_train, missing=-1)
        params = {'booster': 'gbtree',
                  'eval_metric': 'rmse',
                  'gamma': 1,
                  'min_child_weight': 1.5,
                  'max_depth': 5,
                  'lambda': 10,
                  'subsample': 0.7,
                  'colsample_bytree': 0.7,
                  'colsample_bylevel': 0.7,
                  'eta': 0.03,
                  'tree_method': 'exact',
                  'seed': 2017,
                  'nthread': 12
                  }
        num_round = 1000
        early_stopping_rounds = 10
        watchlist = [(train_matrix, 'train'),
                     (test_matrix, 'eval')
                     ]
        model = clf.train(params, train_matrix, num_boost_round=num_round,evals=watchlist,
                          early_stopping_rounds=early_stopping_rounds
                          )
        pre = model.predict(test,ntree_limit=model.best_ntree_limit).reshape(-1,1)
        return pre
    elif clf_fin in ["lgb"]:
        print(clf_name)
        clf = lightgbm
        train_matrix = clf.Dataset(train, label=y_train)
        test_matrix = clf.Dataset(train, label=y_train)
        params = {
                  'boosting_type': 'gbdt',
                  'objective': 'regression_l2',
                  'metric': 'mse',
                  'min_child_weight': 1.5,
                  'num_leaves': 2**5,
                  'lambda_l2': 10,
                  'subsample': 0.7,
                  'colsample_bytree': 0.7,
                  'colsample_bylevel': 0.7,
                  'learning_rate': 0.03,
                  'tree_method': 'exact',
                  'seed': 2017,
                  'nthread': 12,
                  'silent': True,
                  }
        num_round = 1000
        early_stopping_rounds = 10
        model = clf.train(params, train_matrix,num_round,valid_sets=test_matrix,
                          early_stopping_rounds=early_stopping_rounds
                          )
        print('pred')
        pre = model.predict(test,num_iteration=model.best_iteration).reshape(-1,1)
        print(pre)
        return pre

with open("./zhengqi_train.txt")  as fr:
    data_train=pd.read_table(fr,sep="\t")
with open("./zhengqi_test.txt") as fr_test:
    data_test=pd.read_table(fr_test,sep="\t")

### K折交叉验证
from sklearn.model_selection import StratifiedKFold, KFold

folds = 5
seed = 1
kf = KFold(n_splits=5, shuffle=True, random_state=0)

### 训练集和测试集数据
x_train = data_train[data_test.columns].values
x_valid = data_test[data_test.columns].values
y_train = data_train['target'].values

### 使用lr_reg和lgb_reg进行融合预测
clf_list = [lr_reg, lgb_reg]
clf_list = [lr_reg, rf_reg]

#很容易过拟合
pred = stacking_pred(x_train, y_train, x_valid, kf, clf_list, label_split=None, clf_fin="lgb", if_concat_origin=True)
print(pred)

运行结果：

lr now score is: [0.11573216950871248]
lr now score is: [0.11573216950871248, 0.09417486426618929]
lr now score is: [0.11573216950871248, 0.09417486426618929, 0.10805046561851059]
lr now score is: [0.11573216950871248, 0.09417486426618929, 0.10805046561851059, 0.12420887065601556]
lr now score is: [0.11573216950871248, 0.09417486426618929, 0.10805046561851059, 0.12420887065601556, 0.11940113841914012]
lr_score_list: [0.11573216950871248, 0.09417486426618929, 0.10805046561851059, 0.12420887065601556, 0.11940113841914012]
lr_score_mean: 0.1123135016937136
[LightGBM] [Warning] Unknown parameter: tree_method
[LightGBM] [Warning] Unknown parameter: colsample_bylevel
[LightGBM] [Warning] Unknown parameter: silent
[LightGBM] [Warning] Unknown parameter: tree_method
[LightGBM] [Warning] Unknown parameter: colsample_bylevel
[LightGBM] [Warning] Unknown parameter: silent
[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.000775 seconds.
You can set `force_col_wise=true` to remove the overhead.
[LightGBM] [Info] Total Bins 8846
[LightGBM] [Info] Number of data points in the train set: 2310, number of used features: 38
[LightGBM] [Warning] Unknown parameter: tree_method
[LightGBM] [Warning] Unknown parameter: colsample_bylevel
[LightGBM] [Warning] Unknown parameter: silent
[LightGBM] [Info] Start training from score 0.126200
[1]	valid_0's l2: 0.992087
Training until validation scores don't improve for 10 rounds
[2]	valid_0's l2: 0.94735
[3]	valid_0's l2: 0.905091
[4]	valid_0's l2: 0.865237
[5]	valid_0's l2: 0.828478
[6]	valid_0's l2: 0.793015
[7]	valid_0's l2: 0.759026
[8]	valid_0's l2: 0.728042
[9]	valid_0's l2: 0.697924
[10]	valid_0's l2: 0.670335
[11]	valid_0's l2: 0.64293
[12]	valid_0's l2: 0.618124
[13]	valid_0's l2: 0.593382
[14]	valid_0's l2: 0.570581
[15]	valid_0's l2: 0.549105
[16]	valid_0's l2: 0.528357
[17]	valid_0's l2: 0.509096
[18]	valid_0's l2: 0.490868
[19]	valid_0's l2: 0.473523
[20]	valid_0's l2: 0.456674
[21]	valid_0's l2: 0.440983
[22]	valid_0's l2: 0.425928
[23]	valid_0's l2: 0.411831
[24]	valid_0's l2: 0.397973
[25]	valid_0's l2: 0.384988
[26]	valid_0's l2: 0.373082
[27]	valid_0's l2: 0.361415
[28]	valid_0's l2: 0.350399
[29]	valid_0's l2: 0.34
[30]	valid_0's l2: 0.330165
[31]	valid_0's l2: 0.320483
[32]	valid_0's l2: 0.311581
[33]	valid_0's l2: 0.303078
[34]	valid_0's l2: 0.294905
[35]	valid_0's l2: 0.287238
[36]	valid_0's l2: 0.280027
[37]	valid_0's l2: 0.273352
[38]	valid_0's l2: 0.266937
[39]	valid_0's l2: 0.260499
[40]	valid_0's l2: 0.254449
[41]	valid_0's l2: 0.248739
[42]	valid_0's l2: 0.243374
[43]	valid_0's l2: 0.238653
[44]	valid_0's l2: 0.233905
[45]	valid_0's l2: 0.229188
[46]	valid_0's l2: 0.224701
[47]	valid_0's l2: 0.220568
[48]	valid_0's l2: 0.216666
[49]	valid_0's l2: 0.212891
[50]	valid_0's l2: 0.209022
[51]	valid_0's l2: 0.205757
[52]	valid_0's l2: 0.202405
[53]	valid_0's l2: 0.199308
[54]	valid_0's l2: 0.196345
[55]	valid_0's l2: 0.193315
[56]	valid_0's l2: 0.190598
[57]	valid_0's l2: 0.187959
[58]	valid_0's l2: 0.18533
[59]	valid_0's l2: 0.182929
[60]	valid_0's l2: 0.18065
[61]	valid_0's l2: 0.17836
[62]	valid_0's l2: 0.176264
[63]	valid_0's l2: 0.174016
[64]	valid_0's l2: 0.171923
[65]	valid_0's l2: 0.170216
[66]	valid_0's l2: 0.168335
[67]	valid_0's l2: 0.166779
[68]	valid_0's l2: 0.165304
[69]	valid_0's l2: 0.163656
[70]	valid_0's l2: 0.162083
[71]	valid_0's l2: 0.160492
[72]	valid_0's l2: 0.15903
[73]	valid_0's l2: 0.157814
[74]	valid_0's l2: 0.156352
[75]	valid_0's l2: 0.155014
[76]	valid_0's l2: 0.153985
[77]	valid_0's l2: 0.152924
[78]	valid_0's l2: 0.151821
[79]	valid_0's l2: 0.150688
[80]	valid_0's l2: 0.149531
[81]	valid_0's l2: 0.148456
[82]	valid_0's l2: 0.147694
[83]	valid_0's l2: 0.146765
[84]	valid_0's l2: 0.146123
[85]	valid_0's l2: 0.145419
[86]	valid_0's l2: 0.144695
[87]	valid_0's l2: 0.143943
[88]	valid_0's l2: 0.143171
[89]	valid_0's l2: 0.142534
[90]	valid_0's l2: 0.141877
[91]	valid_0's l2: 0.141059
[92]	valid_0's l2: 0.140551
[93]	valid_0's l2: 0.140009
[94]	valid_0's l2: 0.139279
[95]	valid_0's l2: 0.138569
[96]	valid_0's l2: 0.137897
[97]	valid_0's l2: 0.137388
[98]	valid_0's l2: 0.13691
[99]	valid_0's l2: 0.136396
[100]	valid_0's l2: 0.135965
[101]	valid_0's l2: 0.135373
[102]	valid_0's l2: 0.134943
[103]	valid_0's l2: 0.134332
[104]	valid_0's l2: 0.13381
[105]	valid_0's l2: 0.133447
[106]	valid_0's l2: 0.133132
[107]	valid_0's l2: 0.132678
[108]	valid_0's l2: 0.132488
[109]	valid_0's l2: 0.132117
[110]	valid_0's l2: 0.131765
[111]	valid_0's l2: 0.131372
[112]	valid_0's l2: 0.131325
[113]	valid_0's l2: 0.130853
[114]	valid_0's l2: 0.13045
[115]	valid_0's l2: 0.130218
[116]	valid_0's l2: 0.13
[117]	valid_0's l2: 0.129733
[118]	valid_0's l2: 0.129497
[119]	valid_0's l2: 0.129287
[120]	valid_0's l2: 0.128982
[121]	valid_0's l2: 0.128911
[122]	valid_0's l2: 0.128714
[123]	valid_0's l2: 0.128563
[124]	valid_0's l2: 0.128345
[125]	valid_0's l2: 0.12804
[126]	valid_0's l2: 0.127975
[127]	valid_0's l2: 0.127807
[128]	valid_0's l2: 0.127702
[129]	valid_0's l2: 0.127429
[130]	valid_0's l2: 0.127234
[131]	valid_0's l2: 0.127074
[132]	valid_0's l2: 0.127011
[133]	valid_0's l2: 0.12685
[134]	valid_0's l2: 0.12671
[135]	valid_0's l2: 0.126406
[136]	valid_0's l2: 0.126114
[137]	valid_0's l2: 0.125927
[138]	valid_0's l2: 0.125792
[139]	valid_0's l2: 0.125551
[140]	valid_0's l2: 0.125378
[141]	valid_0's l2: 0.125224
[142]	valid_0's l2: 0.125075
[143]	valid_0's l2: 0.124855
[144]	valid_0's l2: 0.124729
[145]	valid_0's l2: 0.124603
[146]	valid_0's l2: 0.124488
[147]	valid_0's l2: 0.124202
[148]	valid_0's l2: 0.123975
[149]	valid_0's l2: 0.123834
[150]	valid_0's l2: 0.123747
[151]	valid_0's l2: 0.123547
[152]	valid_0's l2: 0.123488
[153]	valid_0's l2: 0.123464
[154]	valid_0's l2: 0.123332
[155]	valid_0's l2: 0.123226
[156]	valid_0's l2: 0.123147
[157]	valid_0's l2: 0.123009
[158]	valid_0's l2: 0.122874
[159]	valid_0's l2: 0.122711
[160]	valid_0's l2: 0.122515
[161]	valid_0's l2: 0.12241
[162]	valid_0's l2: 0.122304
[163]	valid_0's l2: 0.122279
[164]	valid_0's l2: 0.122161
[165]	valid_0's l2: 0.122075
[166]	valid_0's l2: 0.121921
[167]	valid_0's l2: 0.121898
[168]	valid_0's l2: 0.121687
[169]	valid_0's l2: 0.121627
[170]	valid_0's l2: 0.121632
[171]	valid_0's l2: 0.121461
[172]	valid_0's l2: 0.12135
[173]	valid_0's l2: 0.121271
[174]	valid_0's l2: 0.1211
[175]	valid_0's l2: 0.121032
[176]	valid_0's l2: 0.12105
[177]	valid_0's l2: 0.120974
[178]	valid_0's l2: 0.120873
[179]	valid_0's l2: 0.120812
[180]	valid_0's l2: 0.120656
[181]	valid_0's l2: 0.120546
[182]	valid_0's l2: 0.120499
[183]	valid_0's l2: 0.12045
[184]	valid_0's l2: 0.120431
[185]	valid_0's l2: 0.120385
[186]	valid_0's l2: 0.120317
[187]	valid_0's l2: 0.120107
[188]	valid_0's l2: 0.120094
[189]	valid_0's l2: 0.120012
[190]	valid_0's l2: 0.119968
[191]	valid_0's l2: 0.119858
[192]	valid_0's l2: 0.119831
[193]	valid_0's l2: 0.119706
[194]	valid_0's l2: 0.119654
[195]	valid_0's l2: 0.119578
[196]	valid_0's l2: 0.119593
[197]	valid_0's l2: 0.119558
[198]	valid_0's l2: 0.119572
[199]	valid_0's l2: 0.11962
[200]	valid_0's l2: 0.119574
[201]	valid_0's l2: 0.119535
[202]	valid_0's l2: 0.119481
[203]	valid_0's l2: 0.1194
[204]	valid_0's l2: 0.119352
[205]	valid_0's l2: 0.119355
[206]	valid_0's l2: 0.119352
[207]	valid_0's l2: 0.119336
[208]	valid_0's l2: 0.119256
[209]	valid_0's l2: 0.119248
[210]	valid_0's l2: 0.1193
[211]	valid_0's l2: 0.119222
[212]	valid_0's l2: 0.1191
[213]	valid_0's l2: 0.119105
[214]	valid_0's l2: 0.119048
[215]	valid_0's l2: 0.119149
[216]	valid_0's l2: 0.119107
[217]	valid_0's l2: 0.119024
[218]	valid_0's l2: 0.118886
[219]	valid_0's l2: 0.118847
[220]	valid_0's l2: 0.118799
[221]	valid_0's l2: 0.118715
[222]	valid_0's l2: 0.11867
[223]	valid_0's l2: 0.118671
[224]	valid_0's l2: 0.118667
[225]	valid_0's l2: 0.118674
[226]	valid_0's l2: 0.118661
[227]	valid_0's l2: 0.118636
[228]	valid_0's l2: 0.118587
[229]	valid_0's l2: 0.118612
[230]	valid_0's l2: 0.118581
[231]	valid_0's l2: 0.118531
[232]	valid_0's l2: 0.118462
[233]	valid_0's l2: 0.118486
[234]	valid_0's l2: 0.118461
[235]	valid_0's l2: 0.11846
[236]	valid_0's l2: 0.118457
[237]	valid_0's l2: 0.118307
[238]	valid_0's l2: 0.118244
[239]	valid_0's l2: 0.118185
[240]	valid_0's l2: 0.11818
[241]	valid_0's l2: 0.118242
[242]	valid_0's l2: 0.118193
[243]	valid_0's l2: 0.118126
[244]	valid_0's l2: 0.118134
[245]	valid_0's l2: 0.118132
[246]	valid_0's l2: 0.11809
[247]	valid_0's l2: 0.118078
[248]	valid_0's l2: 0.118082
[249]	valid_0's l2: 0.118026
[250]	valid_0's l2: 0.117904
[251]	valid_0's l2: 0.117845
[252]	valid_0's l2: 0.11778
[253]	valid_0's l2: 0.117714
[254]	valid_0's l2: 0.117665
[255]	valid_0's l2: 0.117614
[256]	valid_0's l2: 0.117606
[257]	valid_0's l2: 0.117558
[258]	valid_0's l2: 0.117562
[259]	valid_0's l2: 0.117578
[260]	valid_0's l2: 0.117497
[261]	valid_0's l2: 0.117504
[262]	valid_0's l2: 0.117394
[263]	valid_0's l2: 0.117426
[264]	valid_0's l2: 0.117393
[265]	valid_0's l2: 0.117334
[266]	valid_0's l2: 0.117273
[267]	valid_0's l2: 0.117258
[268]	valid_0's l2: 0.117163
[269]	valid_0's l2: 0.117064
[270]	valid_0's l2: 0.117054
[271]	valid_0's l2: 0.116993
[272]	valid_0's l2: 0.116947
[273]	valid_0's l2: 0.116938
[274]	valid_0's l2: 0.11687
[275]	valid_0's l2: 0.116836
[276]	valid_0's l2: 0.116819
[277]	valid_0's l2: 0.116712
[278]	valid_0's l2: 0.116708
[279]	valid_0's l2: 0.116678
[280]	valid_0's l2: 0.116601
[281]	valid_0's l2: 0.116624
[282]	valid_0's l2: 0.116609
[283]	valid_0's l2: 0.116566
[284]	valid_0's l2: 0.116513
[285]	valid_0's l2: 0.116429
[286]	valid_0's l2: 0.116397
[287]	valid_0's l2: 0.116341
[288]	valid_0's l2: 0.116352
[289]	valid_0's l2: 0.116273
[290]	valid_0's l2: 0.116209
[291]	valid_0's l2: 0.116211
[292]	valid_0's l2: 0.116152
[293]	valid_0's l2: 0.116054
[294]	valid_0's l2: 0.116108
[295]	valid_0's l2: 0.116138
[296]	valid_0's l2: 0.116053
[297]	valid_0's l2: 0.115981
[298]	valid_0's l2: 0.115985
[299]	valid_0's l2: 0.115993
[300]	valid_0's l2: 0.116019
[301]	valid_0's l2: 0.115995
[302]	valid_0's l2: 0.115975
[303]	valid_0's l2: 0.116
[304]	valid_0's l2: 0.116
[305]	valid_0's l2: 0.116021
[306]	valid_0's l2: 0.115995
[307]	valid_0's l2: 0.11593
[308]	valid_0's l2: 0.116007
[309]	valid_0's l2: 0.115919
[310]	valid_0's l2: 0.115891
[311]	valid_0's l2: 0.115829
[312]	valid_0's l2: 0.115794
[313]	valid_0's l2: 0.115731
[314]	valid_0's l2: 0.115761
[315]	valid_0's l2: 0.115739
[316]	valid_0's l2: 0.115764
[317]	valid_0's l2: 0.11573
[318]	valid_0's l2: 0.115768
[319]	valid_0's l2: 0.115734
[320]	valid_0's l2: 0.115697
[321]	valid_0's l2: 0.115695
[322]	valid_0's l2: 0.115712
[323]	valid_0's l2: 0.115718
[324]	valid_0's l2: 0.115734
[325]	valid_0's l2: 0.115727
[326]	valid_0's l2: 0.115686
[327]	valid_0's l2: 0.115648
[328]	valid_0's l2: 0.115636
[329]	valid_0's l2: 0.115625
[330]	valid_0's l2: 0.115597
[331]	valid_0's l2: 0.115626
[332]	valid_0's l2: 0.115595
[333]	valid_0's l2: 0.115582
[334]	valid_0's l2: 0.115557
[335]	valid_0's l2: 0.115527
[336]	valid_0's l2: 0.115515
[337]	valid_0's l2: 0.115534
[338]	valid_0's l2: 0.115483
[339]	valid_0's l2: 0.115444
[340]	valid_0's l2: 0.115384
[341]	valid_0's l2: 0.115406
[342]	valid_0's l2: 0.115391
[343]	valid_0's l2: 0.115322
[344]	valid_0's l2: 0.115266
[345]	valid_0's l2: 0.115204
[346]	valid_0's l2: 0.115179
[347]	valid_0's l2: 0.11522
[348]	valid_0's l2: 0.115214
[349]	valid_0's l2: 0.115203
[350]	valid_0's l2: 0.115172
[351]	valid_0's l2: 0.115147
[352]	valid_0's l2: 0.115143
[353]	valid_0's l2: 0.115097
[354]	valid_0's l2: 0.115099
[355]	valid_0's l2: 0.115052
[356]	valid_0's l2: 0.115009
[357]	valid_0's l2: 0.114997
[358]	valid_0's l2: 0.114963
[359]	valid_0's l2: 0.114959
[360]	valid_0's l2: 0.114912
[361]	valid_0's l2: 0.11486
[362]	valid_0's l2: 0.114881
[363]	valid_0's l2: 0.11483
[364]	valid_0's l2: 0.114854
[365]	valid_0's l2: 0.114857
[366]	valid_0's l2: 0.114823
[367]	valid_0's l2: 0.114828
[368]	valid_0's l2: 0.114765
[369]	valid_0's l2: 0.114746
[370]	valid_0's l2: 0.114722
[371]	valid_0's l2: 0.114708
[372]	valid_0's l2: 0.114678
[373]	valid_0's l2: 0.114686
[374]	valid_0's l2: 0.114669
[375]	valid_0's l2: 0.114655
[376]	valid_0's l2: 0.114645
[377]	valid_0's l2: 0.11467
[378]	valid_0's l2: 0.114674
[379]	valid_0's l2: 0.114707
[380]	valid_0's l2: 0.114689
[381]	valid_0's l2: 0.114679
[382]	valid_0's l2: 0.11467
[383]	valid_0's l2: 0.11469
[384]	valid_0's l2: 0.114667
[385]	valid_0's l2: 0.114656
[386]	valid_0's l2: 0.114657
Early stopping, best iteration is:
[376]	valid_0's l2: 0.114645
lgb now score is: [0.11464525315121991]
[LightGBM] [Warning] Unknown parameter: tree_method
[LightGBM] [Warning] Unknown parameter: colsample_bylevel
[LightGBM] [Warning] Unknown parameter: silent
[LightGBM] [Warning] Unknown parameter: tree_method
[LightGBM] [Warning] Unknown parameter: colsample_bylevel
[LightGBM] [Warning] Unknown parameter: silent
[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.000858 seconds.
You can set `force_col_wise=true` to remove the overhead.
[LightGBM] [Info] Total Bins 8833
[LightGBM] [Info] Number of data points in the train set: 2310, number of used features: 38
[LightGBM] [Warning] Unknown parameter: tree_method
[LightGBM] [Warning] Unknown parameter: colsample_bylevel
[LightGBM] [Warning] Unknown parameter: silent
[LightGBM] [Info] Start training from score 0.111094
[1]	valid_0's l2: 0.86091
Training until validation scores don't improve for 10 rounds
[2]	valid_0's l2: 0.822041
[3]	valid_0's l2: 0.784715
[4]	valid_0's l2: 0.749897
[5]	valid_0's l2: 0.71688
[6]	valid_0's l2: 0.686156
[7]	valid_0's l2: 0.656374
[8]	valid_0's l2: 0.628383
[9]	valid_0's l2: 0.601579
[10]	valid_0's l2: 0.576335
[11]	valid_0's l2: 0.552179
[12]	valid_0's l2: 0.529819
[13]	valid_0's l2: 0.508092
[14]	valid_0's l2: 0.488383
[15]	valid_0's l2: 0.46922
[16]	valid_0's l2: 0.45117
[17]	valid_0's l2: 0.434806
[18]	valid_0's l2: 0.419967
[19]	valid_0's l2: 0.404153
[20]	valid_0's l2: 0.389472
[21]	valid_0's l2: 0.375817
[22]	valid_0's l2: 0.362587
[23]	valid_0's l2: 0.349989
[24]	valid_0's l2: 0.337657
[25]	valid_0's l2: 0.326825
[26]	valid_0's l2: 0.316732
[27]	valid_0's l2: 0.306624
[28]	valid_0's l2: 0.296987
[29]	valid_0's l2: 0.287803
[30]	valid_0's l2: 0.279086
[31]	valid_0's l2: 0.27072
[32]	valid_0's l2: 0.262841
[33]	valid_0's l2: 0.255607
[34]	valid_0's l2: 0.24856
[35]	valid_0's l2: 0.24187
[36]	valid_0's l2: 0.23572
[37]	valid_0's l2: 0.229697
[38]	valid_0's l2: 0.223881
[39]	valid_0's l2: 0.218407
[40]	valid_0's l2: 0.213644
[41]	valid_0's l2: 0.208768
[42]	valid_0's l2: 0.204136
[43]	valid_0's l2: 0.199997
[44]	valid_0's l2: 0.196003
[45]	valid_0's l2: 0.192227
[46]	valid_0's l2: 0.18864
[47]	valid_0's l2: 0.185191
[48]	valid_0's l2: 0.181781
[49]	valid_0's l2: 0.178617
[50]	valid_0's l2: 0.17564
[51]	valid_0's l2: 0.17263
[52]	valid_0's l2: 0.170023
[53]	valid_0's l2: 0.167523
[54]	valid_0's l2: 0.16511
[55]	valid_0's l2: 0.162756
[56]	valid_0's l2: 0.160283
[57]	valid_0's l2: 0.158385
[58]	valid_0's l2: 0.156343
[59]	valid_0's l2: 0.154427
[60]	valid_0's l2: 0.152609
[61]	valid_0's l2: 0.150822
[62]	valid_0's l2: 0.148894
[63]	valid_0's l2: 0.147392
[64]	valid_0's l2: 0.145769
[65]	valid_0's l2: 0.14409
[66]	valid_0's l2: 0.142545
[67]	valid_0's l2: 0.141187
[68]	valid_0's l2: 0.139822
[69]	valid_0's l2: 0.138396
[70]	valid_0's l2: 0.137236
[71]	valid_0's l2: 0.136107
[72]	valid_0's l2: 0.135055
[73]	valid_0's l2: 0.133874
[74]	valid_0's l2: 0.132889
[75]	valid_0's l2: 0.131947
[76]	valid_0's l2: 0.131061
[77]	valid_0's l2: 0.130208
[78]	valid_0's l2: 0.12939
[79]	valid_0's l2: 0.128622
[80]	valid_0's l2: 0.127771
[81]	valid_0's l2: 0.126905
[82]	valid_0's l2: 0.126137
[83]	valid_0's l2: 0.12548
[84]	valid_0's l2: 0.124831
[85]	valid_0's l2: 0.12432
[86]	valid_0's l2: 0.123806
[87]	valid_0's l2: 0.12327
[88]	valid_0's l2: 0.122608
[89]	valid_0's l2: 0.12219
[90]	valid_0's l2: 0.12163
[91]	valid_0's l2: 0.121188
[92]	valid_0's l2: 0.120653
[93]	valid_0's l2: 0.120216
[94]	valid_0's l2: 0.119754
[95]	valid_0's l2: 0.119218
[96]	valid_0's l2: 0.118903
[97]	valid_0's l2: 0.118442
[98]	valid_0's l2: 0.118042
[99]	valid_0's l2: 0.117763
[100]	valid_0's l2: 0.117492
[101]	valid_0's l2: 0.117117
[102]	valid_0's l2: 0.116843
[103]	valid_0's l2: 0.116697
[104]	valid_0's l2: 0.116437
[105]	valid_0's l2: 0.11609
[106]	valid_0's l2: 0.115905
[107]	valid_0's l2: 0.115726
[108]	valid_0's l2: 0.115542
[109]	valid_0's l2: 0.115341
[110]	valid_0's l2: 0.115002
[111]	valid_0's l2: 0.11472
[112]	valid_0's l2: 0.114469
[113]	valid_0's l2: 0.114079
[114]	valid_0's l2: 0.113868
[115]	valid_0's l2: 0.113736
[116]	valid_0's l2: 0.113459
[117]	valid_0's l2: 0.113129
[118]	valid_0's l2: 0.112894
[119]	valid_0's l2: 0.112745
[120]	valid_0's l2: 0.112488
[121]	valid_0's l2: 0.112348
[122]	valid_0's l2: 0.112149
[123]	valid_0's l2: 0.11209
[124]	valid_0's l2: 0.112012
[125]	valid_0's l2: 0.111745
[126]	valid_0's l2: 0.111642
[127]	valid_0's l2: 0.111644
[128]	valid_0's l2: 0.111437
[129]	valid_0's l2: 0.111257
[130]	valid_0's l2: 0.111145
[131]	valid_0's l2: 0.110934
[132]	valid_0's l2: 0.110702
[133]	valid_0's l2: 0.110584
[134]	valid_0's l2: 0.11045
[135]	valid_0's l2: 0.110237
[136]	valid_0's l2: 0.110031
[137]	valid_0's l2: 0.109968
[138]	valid_0's l2: 0.109877
[139]	valid_0's l2: 0.109718
[140]	valid_0's l2: 0.109607
[141]	valid_0's l2: 0.109424
[142]	valid_0's l2: 0.109276
[143]	valid_0's l2: 0.109145
[144]	valid_0's l2: 0.10913
[145]	valid_0's l2: 0.108893
[146]	valid_0's l2: 0.108787
[147]	valid_0's l2: 0.10868
[148]	valid_0's l2: 0.108603
[149]	valid_0's l2: 0.108512
[150]	valid_0's l2: 0.108397
[151]	valid_0's l2: 0.108256
[152]	valid_0's l2: 0.108152
[153]	valid_0's l2: 0.108033
[154]	valid_0's l2: 0.10799
[155]	valid_0's l2: 0.107907
[156]	valid_0's l2: 0.107812
[157]	valid_0's l2: 0.107647
[158]	valid_0's l2: 0.107671
[159]	valid_0's l2: 0.107591
[160]	valid_0's l2: 0.107406
[161]	valid_0's l2: 0.107304
[162]	valid_0's l2: 0.107164
[163]	valid_0's l2: 0.107023
[164]	valid_0's l2: 0.106946
[165]	valid_0's l2: 0.106876
[166]	valid_0's l2: 0.106805
[167]	valid_0's l2: 0.106705
[168]	valid_0's l2: 0.106638
[169]	valid_0's l2: 0.106514
[170]	valid_0's l2: 0.106384
[171]	valid_0's l2: 0.106304
[172]	valid_0's l2: 0.106133
[173]	valid_0's l2: 0.106015
[174]	valid_0's l2: 0.105924
[175]	valid_0's l2: 0.105863
[176]	valid_0's l2: 0.105869
[177]	valid_0's l2: 0.105779
[178]	valid_0's l2: 0.105551
[179]	valid_0's l2: 0.105415
[180]	valid_0's l2: 0.105419
[181]	valid_0's l2: 0.105338
[182]	valid_0's l2: 0.105306
[183]	valid_0's l2: 0.105239
[184]	valid_0's l2: 0.105156
[185]	valid_0's l2: 0.105091
[186]	valid_0's l2: 0.104991
[187]	valid_0's l2: 0.104883
[188]	valid_0's l2: 0.104786
[189]	valid_0's l2: 0.104672
[190]	valid_0's l2: 0.104588
[191]	valid_0's l2: 0.10446
[192]	valid_0's l2: 0.104321
[193]	valid_0's l2: 0.104167
[194]	valid_0's l2: 0.104112
[195]	valid_0's l2: 0.104088
[196]	valid_0's l2: 0.103962
[197]	valid_0's l2: 0.103874
[198]	valid_0's l2: 0.103831
[199]	valid_0's l2: 0.10369
[200]	valid_0's l2: 0.103604
[201]	valid_0's l2: 0.103497
[202]	valid_0's l2: 0.103407
[203]	valid_0's l2: 0.103347
[204]	valid_0's l2: 0.103283
[205]	valid_0's l2: 0.103201
[206]	valid_0's l2: 0.103152
[207]	valid_0's l2: 0.103015
[208]	valid_0's l2: 0.10293
[209]	valid_0's l2: 0.102952
[210]	valid_0's l2: 0.102864
[211]	valid_0's l2: 0.102785
[212]	valid_0's l2: 0.102692
[213]	valid_0's l2: 0.102638
[214]	valid_0's l2: 0.102561
[215]	valid_0's l2: 0.10256
[216]	valid_0's l2: 0.102528
[217]	valid_0's l2: 0.102487
[218]	valid_0's l2: 0.102494
[219]	valid_0's l2: 0.10243
[220]	valid_0's l2: 0.102457
[221]	valid_0's l2: 0.102365
[222]	valid_0's l2: 0.102228
[223]	valid_0's l2: 0.102156
[224]	valid_0's l2: 0.102063
[225]	valid_0's l2: 0.102145
[226]	valid_0's l2: 0.102083
[227]	valid_0's l2: 0.102066
[228]	valid_0's l2: 0.102016
[229]	valid_0's l2: 0.102006
[230]	valid_0's l2: 0.101921
[231]	valid_0's l2: 0.101862
[232]	valid_0's l2: 0.101847
[233]	valid_0's l2: 0.101761
[234]	valid_0's l2: 0.101755
[235]	valid_0's l2: 0.101747
[236]	valid_0's l2: 0.101702
[237]	valid_0's l2: 0.101608
[238]	valid_0's l2: 0.101607
[239]	valid_0's l2: 0.101464
[240]	valid_0's l2: 0.101384
[241]	valid_0's l2: 0.101364
[242]	valid_0's l2: 0.101346
[243]	valid_0's l2: 0.101302
[244]	valid_0's l2: 0.101225
[245]	valid_0's l2: 0.101195
[246]	valid_0's l2: 0.101125
[247]	valid_0's l2: 0.101072
[248]	valid_0's l2: 0.101005
[249]	valid_0's l2: 0.100976
[250]	valid_0's l2: 0.100874
[251]	valid_0's l2: 0.100874
[252]	valid_0's l2: 0.100843
[253]	valid_0's l2: 0.100802
[254]	valid_0's l2: 0.100783
[255]	valid_0's l2: 0.100779
[256]	valid_0's l2: 0.100681
[257]	valid_0's l2: 0.10069
[258]	valid_0's l2: 0.100656
[259]	valid_0's l2: 0.100587
[260]	valid_0's l2: 0.100539
[261]	valid_0's l2: 0.100492
[262]	valid_0's l2: 0.10048
[263]	valid_0's l2: 0.100485
[264]	valid_0's l2: 0.100487
[265]	valid_0's l2: 0.100397
[266]	valid_0's l2: 0.100363
[267]	valid_0's l2: 0.100408
[268]	valid_0's l2: 0.100387
[269]	valid_0's l2: 0.100363
[270]	valid_0's l2: 0.100378
[271]	valid_0's l2: 0.100295
[272]	valid_0's l2: 0.100238
[273]	valid_0's l2: 0.100259
[274]	valid_0's l2: 0.100276
[275]	valid_0's l2: 0.100261
[276]	valid_0's l2: 0.100274
[277]	valid_0's l2: 0.100225
[278]	valid_0's l2: 0.100204
[279]	valid_0's l2: 0.10016
[280]	valid_0's l2: 0.100246
[281]	valid_0's l2: 0.100259
[282]	valid_0's l2: 0.100302
[283]	valid_0's l2: 0.100283
[284]	valid_0's l2: 0.100334
[285]	valid_0's l2: 0.100346
[286]	valid_0's l2: 0.100354
[287]	valid_0's l2: 0.100367
[288]	valid_0's l2: 0.100362
[289]	valid_0's l2: 0.100308
Early stopping, best iteration is:
[279]	valid_0's l2: 0.10016
lgb now score is: [0.11464525315121991, 0.1001602219958722]
[LightGBM] [Warning] Unknown parameter: tree_method
[LightGBM] [Warning] Unknown parameter: colsample_bylevel
[LightGBM] [Warning] Unknown parameter: silent
[LightGBM] [Warning] Unknown parameter: tree_method
[LightGBM] [Warning] Unknown parameter: colsample_bylevel
[LightGBM] [Warning] Unknown parameter: silent
[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.000817 seconds.
You can set `force_col_wise=true` to remove the overhead.
[LightGBM] [Info] Total Bins 8847
[LightGBM] [Info] Number of data points in the train set: 2310, number of used features: 38
[LightGBM] [Warning] Unknown parameter: tree_method
[LightGBM] [Warning] Unknown parameter: colsample_bylevel
[LightGBM] [Warning] Unknown parameter: silent
[LightGBM] [Info] Start training from score 0.137391
[1]	valid_0's l2: 0.925026
Training until validation scores don't improve for 10 rounds
[2]	valid_0's l2: 0.882263
[3]	valid_0's l2: 0.841261
[4]	valid_0's l2: 0.802716
[5]	valid_0's l2: 0.766211
[6]	valid_0's l2: 0.732444
[7]	valid_0's l2: 0.700039
[8]	valid_0's l2: 0.669955
[9]	valid_0's l2: 0.641029
[10]	valid_0's l2: 0.614484
[11]	valid_0's l2: 0.588721
[12]	valid_0's l2: 0.564669
[13]	valid_0's l2: 0.541491
[14]	valid_0's l2: 0.519434
[15]	valid_0's l2: 0.498759
[16]	valid_0's l2: 0.478783
[17]	valid_0's l2: 0.460135
[18]	valid_0's l2: 0.443683
[19]	valid_0's l2: 0.427316
[20]	valid_0's l2: 0.411839
[21]	valid_0's l2: 0.396345
[22]	valid_0's l2: 0.382314
[23]	valid_0's l2: 0.368991
[24]	valid_0's l2: 0.356666
[25]	valid_0's l2: 0.344728
[26]	valid_0's l2: 0.332818
[27]	valid_0's l2: 0.322327
[28]	valid_0's l2: 0.312361
[29]	valid_0's l2: 0.302618
[30]	valid_0's l2: 0.293583
[31]	valid_0's l2: 0.284772
[32]	valid_0's l2: 0.27643
[33]	valid_0's l2: 0.268242
[34]	valid_0's l2: 0.260699
[35]	valid_0's l2: 0.25392
[36]	valid_0's l2: 0.247676
[37]	valid_0's l2: 0.241242
[38]	valid_0's l2: 0.235133
[39]	valid_0's l2: 0.229733
[40]	valid_0's l2: 0.224359
[41]	valid_0's l2: 0.219506
[42]	valid_0's l2: 0.214509
[43]	valid_0's l2: 0.210222
[44]	valid_0's l2: 0.205951
[45]	valid_0's l2: 0.201601
[46]	valid_0's l2: 0.197821
[47]	valid_0's l2: 0.193874
[48]	valid_0's l2: 0.190225
[49]	valid_0's l2: 0.1869
[50]	valid_0's l2: 0.183644
[51]	valid_0's l2: 0.180409
[52]	valid_0's l2: 0.177691
[53]	valid_0's l2: 0.174935
[54]	valid_0's l2: 0.172226
[55]	valid_0's l2: 0.169595
[56]	valid_0's l2: 0.167186
[57]	valid_0's l2: 0.164992
[58]	valid_0's l2: 0.162876
[59]	valid_0's l2: 0.16082
[60]	valid_0's l2: 0.158811
[61]	valid_0's l2: 0.156882
[62]	valid_0's l2: 0.155007
[63]	valid_0's l2: 0.153246
[64]	valid_0's l2: 0.151493
[65]	valid_0's l2: 0.149942
[66]	valid_0's l2: 0.148382
[67]	valid_0's l2: 0.146962
[68]	valid_0's l2: 0.14547
[69]	valid_0's l2: 0.144072
[70]	valid_0's l2: 0.142969
[71]	valid_0's l2: 0.141822
[72]	valid_0's l2: 0.140529
[73]	valid_0's l2: 0.139502
[74]	valid_0's l2: 0.138325
[75]	valid_0's l2: 0.13723
[76]	valid_0's l2: 0.136285
[77]	valid_0's l2: 0.13545
[78]	valid_0's l2: 0.134449
[79]	valid_0's l2: 0.13355
[80]	valid_0's l2: 0.13274
[81]	valid_0's l2: 0.132011
[82]	valid_0's l2: 0.131104
[83]	valid_0's l2: 0.130412
[84]	valid_0's l2: 0.129725
[85]	valid_0's l2: 0.129029
[86]	valid_0's l2: 0.12832
[87]	valid_0's l2: 0.127588
[88]	valid_0's l2: 0.126903
[89]	valid_0's l2: 0.126178
[90]	valid_0's l2: 0.125507
[91]	valid_0's l2: 0.12504
[92]	valid_0's l2: 0.124611
[93]	valid_0's l2: 0.124076
[94]	valid_0's l2: 0.123501
[95]	valid_0's l2: 0.122931
[96]	valid_0's l2: 0.122337
[97]	valid_0's l2: 0.121932
[98]	valid_0's l2: 0.121426
[99]	valid_0's l2: 0.121102
[100]	valid_0's l2: 0.12074
[101]	valid_0's l2: 0.12041
[102]	valid_0's l2: 0.119975
[103]	valid_0's l2: 0.119506
[104]	valid_0's l2: 0.119246
[105]	valid_0's l2: 0.119108
[106]	valid_0's l2: 0.118802
[107]	valid_0's l2: 0.118554
[108]	valid_0's l2: 0.118359
[109]	valid_0's l2: 0.118068
[110]	valid_0's l2: 0.117868
[111]	valid_0's l2: 0.117693
[112]	valid_0's l2: 0.117375
[113]	valid_0's l2: 0.117295
[114]	valid_0's l2: 0.117114
[115]	valid_0's l2: 0.116833
[116]	valid_0's l2: 0.116496
[117]	valid_0's l2: 0.116232
[118]	valid_0's l2: 0.115975
[119]	valid_0's l2: 0.115697
[120]	valid_0's l2: 0.115323
[121]	valid_0's l2: 0.114989
[122]	valid_0's l2: 0.114795
[123]	valid_0's l2: 0.114335
[124]	valid_0's l2: 0.114077
[125]	valid_0's l2: 0.11381
[126]	valid_0's l2: 0.113463
[127]	valid_0's l2: 0.113259
[128]	valid_0's l2: 0.113066
[129]	valid_0's l2: 0.112829
[130]	valid_0's l2: 0.112548
[131]	valid_0's l2: 0.112225
[132]	valid_0's l2: 0.112089
[133]	valid_0's l2: 0.111891
[134]	valid_0's l2: 0.111786
[135]	valid_0's l2: 0.111564
[136]	valid_0's l2: 0.111354
[137]	valid_0's l2: 0.111186
[138]	valid_0's l2: 0.110975
[139]	valid_0's l2: 0.110834
[140]	valid_0's l2: 0.110615
[141]	valid_0's l2: 0.110417
[142]	valid_0's l2: 0.110173
[143]	valid_0's l2: 0.109991
[144]	valid_0's l2: 0.109864
[145]	valid_0's l2: 0.109838
[146]	valid_0's l2: 0.109655
[147]	valid_0's l2: 0.109553
[148]	valid_0's l2: 0.109457
[149]	valid_0's l2: 0.109316
[150]	valid_0's l2: 0.109164
[151]	valid_0's l2: 0.10898
[152]	valid_0's l2: 0.108961
[153]	valid_0's l2: 0.108942
[154]	valid_0's l2: 0.10879
[155]	valid_0's l2: 0.10868
[156]	valid_0's l2: 0.108643
[157]	valid_0's l2: 0.108405
[158]	valid_0's l2: 0.108422
[159]	valid_0's l2: 0.10839
[160]	valid_0's l2: 0.108262
[161]	valid_0's l2: 0.108223
[162]	valid_0's l2: 0.108148
[163]	valid_0's l2: 0.10811
[164]	valid_0's l2: 0.107988
[165]	valid_0's l2: 0.107869
[166]	valid_0's l2: 0.107822
[167]	valid_0's l2: 0.107675
[168]	valid_0's l2: 0.107646
[169]	valid_0's l2: 0.107497
[170]	valid_0's l2: 0.107425
[171]	valid_0's l2: 0.107329
[172]	valid_0's l2: 0.107197
[173]	valid_0's l2: 0.107198
[174]	valid_0's l2: 0.107122
[175]	valid_0's l2: 0.107053
[176]	valid_0's l2: 0.106904
[177]	valid_0's l2: 0.10679
[178]	valid_0's l2: 0.106755
[179]	valid_0's l2: 0.106669
[180]	valid_0's l2: 0.10663
[181]	valid_0's l2: 0.106529
[182]	valid_0's l2: 0.106497
[183]	valid_0's l2: 0.106452
[184]	valid_0's l2: 0.106409
[185]	valid_0's l2: 0.106313
[186]	valid_0's l2: 0.106197
[187]	valid_0's l2: 0.106146
[188]	valid_0's l2: 0.106034
[189]	valid_0's l2: 0.10599
[190]	valid_0's l2: 0.105846
[191]	valid_0's l2: 0.105816
[192]	valid_0's l2: 0.105789
[193]	valid_0's l2: 0.105665
[194]	valid_0's l2: 0.105676
[195]	valid_0's l2: 0.105641
[196]	valid_0's l2: 0.105541
[197]	valid_0's l2: 0.105433
[198]	valid_0's l2: 0.105324
[199]	valid_0's l2: 0.105224
[200]	valid_0's l2: 0.105179
[201]	valid_0's l2: 0.105183
[202]	valid_0's l2: 0.105147
[203]	valid_0's l2: 0.105078
[204]	valid_0's l2: 0.105004
[205]	valid_0's l2: 0.104887
[206]	valid_0's l2: 0.104842
[207]	valid_0's l2: 0.104784
[208]	valid_0's l2: 0.104785
[209]	valid_0's l2: 0.104708
[210]	valid_0's l2: 0.104679
[211]	valid_0's l2: 0.104632
[212]	valid_0's l2: 0.104641
[213]	valid_0's l2: 0.104635
[214]	valid_0's l2: 0.104577
[215]	valid_0's l2: 0.104529
[216]	valid_0's l2: 0.104524
[217]	valid_0's l2: 0.104494
[218]	valid_0's l2: 0.104406
[219]	valid_0's l2: 0.104381
[220]	valid_0's l2: 0.104399
[221]	valid_0's l2: 0.104383
[222]	valid_0's l2: 0.104298
[223]	valid_0's l2: 0.104221
[224]	valid_0's l2: 0.1042
[225]	valid_0's l2: 0.104102
[226]	valid_0's l2: 0.1041
[227]	valid_0's l2: 0.104083
[228]	valid_0's l2: 0.104095
[229]	valid_0's l2: 0.104039
[230]	valid_0's l2: 0.104092
[231]	valid_0's l2: 0.104071
[232]	valid_0's l2: 0.103929
[233]	valid_0's l2: 0.103913
[234]	valid_0's l2: 0.103853
[235]	valid_0's l2: 0.103845
[236]	valid_0's l2: 0.103753
[237]	valid_0's l2: 0.103843
[238]	valid_0's l2: 0.103845
[239]	valid_0's l2: 0.103858
[240]	valid_0's l2: 0.103875
[241]	valid_0's l2: 0.103815
[242]	valid_0's l2: 0.103871
[243]	valid_0's l2: 0.103868
[244]	valid_0's l2: 0.103859
[245]	valid_0's l2: 0.103853
[246]	valid_0's l2: 0.103829
Early stopping, best iteration is:
[236]	valid_0's l2: 0.103753
lgb now score is: [0.11464525315121991, 0.1001602219958722, 0.10375315940652878]
[LightGBM] [Warning] Unknown parameter: tree_method
[LightGBM] [Warning] Unknown parameter: colsample_bylevel
[LightGBM] [Warning] Unknown parameter: silent
[LightGBM] [Warning] Unknown parameter: tree_method
[LightGBM] [Warning] Unknown parameter: colsample_bylevel
[LightGBM] [Warning] Unknown parameter: silent
[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.000785 seconds.
You can set `force_col_wise=true` to remove the overhead.
[LightGBM] [Info] Total Bins 8852
[LightGBM] [Info] Number of data points in the train set: 2311, number of used features: 38
[LightGBM] [Warning] Unknown parameter: tree_method
[LightGBM] [Warning] Unknown parameter: colsample_bylevel
[LightGBM] [Warning] Unknown parameter: silent
[LightGBM] [Info] Start training from score 0.126265
[1]	valid_0's l2: 0.908704
Training until validation scores don't improve for 10 rounds
[2]	valid_0's l2: 0.868691
[3]	valid_0's l2: 0.830532
[4]	valid_0's l2: 0.794315
[5]	valid_0's l2: 0.7611
[6]	valid_0's l2: 0.728347
[7]	valid_0's l2: 0.698418
[8]	valid_0's l2: 0.669822
[9]	valid_0's l2: 0.642236
[10]	valid_0's l2: 0.617694
[11]	valid_0's l2: 0.593604
[12]	valid_0's l2: 0.57053
[13]	valid_0's l2: 0.548345
[14]	valid_0's l2: 0.527479
[15]	valid_0's l2: 0.508192
[16]	valid_0's l2: 0.489393
[17]	valid_0's l2: 0.471802
[18]	valid_0's l2: 0.456568
[19]	valid_0's l2: 0.440842
[20]	valid_0's l2: 0.425425
[21]	valid_0's l2: 0.411385
[22]	valid_0's l2: 0.39851
[23]	valid_0's l2: 0.3862
[24]	valid_0's l2: 0.374549
[25]	valid_0's l2: 0.363075
[26]	valid_0's l2: 0.352338
[27]	valid_0's l2: 0.342305
[28]	valid_0's l2: 0.332717
[29]	valid_0's l2: 0.323468
[30]	valid_0's l2: 0.314911
[31]	valid_0's l2: 0.306372
[32]	valid_0's l2: 0.298758
[33]	valid_0's l2: 0.291218
[34]	valid_0's l2: 0.284186
[35]	valid_0's l2: 0.277231
[36]	valid_0's l2: 0.270833
[37]	valid_0's l2: 0.264518
[38]	valid_0's l2: 0.258655
[39]	valid_0's l2: 0.253038
[40]	valid_0's l2: 0.247684
[41]	valid_0's l2: 0.242439
[42]	valid_0's l2: 0.23761
[43]	valid_0's l2: 0.233467
[44]	valid_0's l2: 0.229177
[45]	valid_0's l2: 0.224878
[46]	valid_0's l2: 0.220847
[47]	valid_0's l2: 0.216928
[48]	valid_0's l2: 0.213423
[49]	valid_0's l2: 0.209869
[50]	valid_0's l2: 0.206618
[51]	valid_0's l2: 0.203104
[52]	valid_0's l2: 0.200198
[53]	valid_0's l2: 0.197226
[54]	valid_0's l2: 0.194248
[55]	valid_0's l2: 0.19163
[56]	valid_0's l2: 0.188989
[57]	valid_0's l2: 0.186768
[58]	valid_0's l2: 0.184316
[59]	valid_0's l2: 0.181846
[60]	valid_0's l2: 0.179628
[61]	valid_0's l2: 0.177584
[62]	valid_0's l2: 0.175533
[63]	valid_0's l2: 0.173613
[64]	valid_0's l2: 0.17185
[65]	valid_0's l2: 0.170217
[66]	valid_0's l2: 0.168476
[67]	valid_0's l2: 0.166895
[68]	valid_0's l2: 0.165221
[69]	valid_0's l2: 0.163738
[70]	valid_0's l2: 0.162531
[71]	valid_0's l2: 0.161344
[72]	valid_0's l2: 0.160004
[73]	valid_0's l2: 0.158696
[74]	valid_0's l2: 0.157288
[75]	valid_0's l2: 0.155998
[76]	valid_0's l2: 0.15503
[77]	valid_0's l2: 0.1542
[78]	valid_0's l2: 0.153279
[79]	valid_0's l2: 0.152067
[80]	valid_0's l2: 0.15107
[81]	valid_0's l2: 0.150341
[82]	valid_0's l2: 0.149716
[83]	valid_0's l2: 0.148821
[84]	valid_0's l2: 0.148028
[85]	valid_0's l2: 0.147276
[86]	valid_0's l2: 0.146672
[87]	valid_0's l2: 0.145871
[88]	valid_0's l2: 0.145176
[89]	valid_0's l2: 0.144463
[90]	valid_0's l2: 0.143833
[91]	valid_0's l2: 0.143094
[92]	valid_0's l2: 0.142517
[93]	valid_0's l2: 0.141894
[94]	valid_0's l2: 0.141182
[95]	valid_0's l2: 0.140756
[96]	valid_0's l2: 0.140221
[97]	valid_0's l2: 0.139578
[98]	valid_0's l2: 0.139056
[99]	valid_0's l2: 0.138617
[100]	valid_0's l2: 0.137901
[101]	valid_0's l2: 0.137304
[102]	valid_0's l2: 0.13684
[103]	valid_0's l2: 0.13648
[104]	valid_0's l2: 0.13598
[105]	valid_0's l2: 0.135534
[106]	valid_0's l2: 0.135175
[107]	valid_0's l2: 0.134796
[108]	valid_0's l2: 0.134354
[109]	valid_0's l2: 0.13388
[110]	valid_0's l2: 0.13349
[111]	valid_0's l2: 0.133151
[112]	valid_0's l2: 0.132774
[113]	valid_0's l2: 0.132428
[114]	valid_0's l2: 0.132023
[115]	valid_0's l2: 0.131581
[116]	valid_0's l2: 0.131294
[117]	valid_0's l2: 0.131037
[118]	valid_0's l2: 0.130764
[119]	valid_0's l2: 0.130524
[120]	valid_0's l2: 0.130234
[121]	valid_0's l2: 0.130037
[122]	valid_0's l2: 0.129779
[123]	valid_0's l2: 0.129572
[124]	valid_0's l2: 0.129269
[125]	valid_0's l2: 0.12902
[126]	valid_0's l2: 0.128784
[127]	valid_0's l2: 0.128563
[128]	valid_0's l2: 0.128316
[129]	valid_0's l2: 0.128097
[130]	valid_0's l2: 0.127928
[131]	valid_0's l2: 0.127674
[132]	valid_0's l2: 0.127558
[133]	valid_0's l2: 0.127249
[134]	valid_0's l2: 0.127054
[135]	valid_0's l2: 0.126784
[136]	valid_0's l2: 0.126564
[137]	valid_0's l2: 0.126427
[138]	valid_0's l2: 0.126325
[139]	valid_0's l2: 0.126169
[140]	valid_0's l2: 0.125961
[141]	valid_0's l2: 0.125765
[142]	valid_0's l2: 0.125582
[143]	valid_0's l2: 0.125344
[144]	valid_0's l2: 0.125255
[145]	valid_0's l2: 0.125064
[146]	valid_0's l2: 0.12491
[147]	valid_0's l2: 0.124751
[148]	valid_0's l2: 0.124617
[149]	valid_0's l2: 0.124444
[150]	valid_0's l2: 0.12425
[151]	valid_0's l2: 0.124184
[152]	valid_0's l2: 0.123996
[153]	valid_0's l2: 0.123928
[154]	valid_0's l2: 0.123756
[155]	valid_0's l2: 0.123536
[156]	valid_0's l2: 0.123461
[157]	valid_0's l2: 0.123334
[158]	valid_0's l2: 0.123178
[159]	valid_0's l2: 0.122978
[160]	valid_0's l2: 0.122906
[161]	valid_0's l2: 0.122794
[162]	valid_0's l2: 0.122735
[163]	valid_0's l2: 0.122631
[164]	valid_0's l2: 0.122532
[165]	valid_0's l2: 0.122428
[166]	valid_0's l2: 0.122406
[167]	valid_0's l2: 0.122365
[168]	valid_0's l2: 0.122219
[169]	valid_0's l2: 0.12221
[170]	valid_0's l2: 0.122085
[171]	valid_0's l2: 0.122005
[172]	valid_0's l2: 0.121934
[173]	valid_0's l2: 0.121767
[174]	valid_0's l2: 0.121818
[175]	valid_0's l2: 0.121747
[176]	valid_0's l2: 0.121677
[177]	valid_0's l2: 0.121591
[178]	valid_0's l2: 0.121582
[179]	valid_0's l2: 0.121484
[180]	valid_0's l2: 0.1214
[181]	valid_0's l2: 0.121248
[182]	valid_0's l2: 0.12112
[183]	valid_0's l2: 0.121066
[184]	valid_0's l2: 0.120999
[185]	valid_0's l2: 0.120897
[186]	valid_0's l2: 0.120918
[187]	valid_0's l2: 0.120862
[188]	valid_0's l2: 0.120789
[189]	valid_0's l2: 0.12073
[190]	valid_0's l2: 0.120684
[191]	valid_0's l2: 0.120622
[192]	valid_0's l2: 0.120585
[193]	valid_0's l2: 0.120482
[194]	valid_0's l2: 0.12043
[195]	valid_0's l2: 0.120443
[196]	valid_0's l2: 0.120366
[197]	valid_0's l2: 0.120223
[198]	valid_0's l2: 0.120173
[199]	valid_0's l2: 0.120144
[200]	valid_0's l2: 0.120074
[201]	valid_0's l2: 0.120014
[202]	valid_0's l2: 0.119923
[203]	valid_0's l2: 0.119829
[204]	valid_0's l2: 0.119771
[205]	valid_0's l2: 0.119675
[206]	valid_0's l2: 0.11956
[207]	valid_0's l2: 0.119615
[208]	valid_0's l2: 0.119598
[209]	valid_0's l2: 0.119612
[210]	valid_0's l2: 0.119571
[211]	valid_0's l2: 0.119534
[212]	valid_0's l2: 0.119483
[213]	valid_0's l2: 0.119477
[214]	valid_0's l2: 0.119439
[215]	valid_0's l2: 0.119413
[216]	valid_0's l2: 0.119368
[217]	valid_0's l2: 0.119304
[218]	valid_0's l2: 0.119201
[219]	valid_0's l2: 0.119133
[220]	valid_0's l2: 0.119044
[221]	valid_0's l2: 0.11902
[222]	valid_0's l2: 0.118855
[223]	valid_0's l2: 0.118823
[224]	valid_0's l2: 0.118805
[225]	valid_0's l2: 0.118776
[226]	valid_0's l2: 0.118777
[227]	valid_0's l2: 0.118727
[228]	valid_0's l2: 0.118676
[229]	valid_0's l2: 0.118682
[230]	valid_0's l2: 0.118591
[231]	valid_0's l2: 0.118531
[232]	valid_0's l2: 0.118467
[233]	valid_0's l2: 0.118464
[234]	valid_0's l2: 0.118355
[235]	valid_0's l2: 0.118341
[236]	valid_0's l2: 0.118404
[237]	valid_0's l2: 0.118371
[238]	valid_0's l2: 0.118304
[239]	valid_0's l2: 0.118237
[240]	valid_0's l2: 0.118115
[241]	valid_0's l2: 0.11802
[242]	valid_0's l2: 0.118064
[243]	valid_0's l2: 0.118044
[244]	valid_0's l2: 0.118033
[245]	valid_0's l2: 0.117901
[246]	valid_0's l2: 0.117935
[247]	valid_0's l2: 0.117907
[248]	valid_0's l2: 0.117851
[249]	valid_0's l2: 0.117779
[250]	valid_0's l2: 0.117733
[251]	valid_0's l2: 0.117687
[252]	valid_0's l2: 0.1177
[253]	valid_0's l2: 0.117634
[254]	valid_0's l2: 0.117604
[255]	valid_0's l2: 0.117523
[256]	valid_0's l2: 0.117526
[257]	valid_0's l2: 0.117537
[258]	valid_0's l2: 0.117514
[259]	valid_0's l2: 0.117486
[260]	valid_0's l2: 0.117431
[261]	valid_0's l2: 0.117423
[262]	valid_0's l2: 0.117415
[263]	valid_0's l2: 0.117318
[264]	valid_0's l2: 0.117347
[265]	valid_0's l2: 0.117354
[266]	valid_0's l2: 0.117369
[267]	valid_0's l2: 0.117323
[268]	valid_0's l2: 0.117319
[269]	valid_0's l2: 0.117312
[270]	valid_0's l2: 0.117242
[271]	valid_0's l2: 0.117219
[272]	valid_0's l2: 0.117197
[273]	valid_0's l2: 0.117206
[274]	valid_0's l2: 0.117185
[275]	valid_0's l2: 0.117144
[276]	valid_0's l2: 0.117173
[277]	valid_0's l2: 0.117149
[278]	valid_0's l2: 0.117094
[279]	valid_0's l2: 0.117011
[280]	valid_0's l2: 0.117057
[281]	valid_0's l2: 0.117051
[282]	valid_0's l2: 0.11699
[283]	valid_0's l2: 0.116972
[284]	valid_0's l2: 0.116944
[285]	valid_0's l2: 0.116927
[286]	valid_0's l2: 0.116896
[287]	valid_0's l2: 0.116851
[288]	valid_0's l2: 0.116828
[289]	valid_0's l2: 0.116835
[290]	valid_0's l2: 0.116745
[291]	valid_0's l2: 0.116734
[292]	valid_0's l2: 0.116606
[293]	valid_0's l2: 0.116575
[294]	valid_0's l2: 0.116496
[295]	valid_0's l2: 0.116509
[296]	valid_0's l2: 0.116569
[297]	valid_0's l2: 0.11651
[298]	valid_0's l2: 0.116459
[299]	valid_0's l2: 0.116448
[300]	valid_0's l2: 0.116377
[301]	valid_0's l2: 0.116332
[302]	valid_0's l2: 0.116302
[303]	valid_0's l2: 0.116206
[304]	valid_0's l2: 0.116181
[305]	valid_0's l2: 0.11621
[306]	valid_0's l2: 0.116195
[307]	valid_0's l2: 0.116138
[308]	valid_0's l2: 0.116128
[309]	valid_0's l2: 0.116061
[310]	valid_0's l2: 0.116021
[311]	valid_0's l2: 0.116014
[312]	valid_0's l2: 0.11599
[313]	valid_0's l2: 0.115987
[314]	valid_0's l2: 0.115936
[315]	valid_0's l2: 0.115952
[316]	valid_0's l2: 0.115952
[317]	valid_0's l2: 0.11591
[318]	valid_0's l2: 0.115905
[319]	valid_0's l2: 0.115813
[320]	valid_0's l2: 0.115804
[321]	valid_0's l2: 0.115803
[322]	valid_0's l2: 0.115789
[323]	valid_0's l2: 0.115777
[324]	valid_0's l2: 0.115737
[325]	valid_0's l2: 0.11575
[326]	valid_0's l2: 0.115736
[327]	valid_0's l2: 0.115735
[328]	valid_0's l2: 0.115712
[329]	valid_0's l2: 0.115731
[330]	valid_0's l2: 0.115698
[331]	valid_0's l2: 0.115667
[332]	valid_0's l2: 0.115652
[333]	valid_0's l2: 0.11564
[334]	valid_0's l2: 0.115639
[335]	valid_0's l2: 0.115645
[336]	valid_0's l2: 0.115679
[337]	valid_0's l2: 0.115679
[338]	valid_0's l2: 0.115659
[339]	valid_0's l2: 0.115693
[340]	valid_0's l2: 0.115663
[341]	valid_0's l2: 0.115645
[342]	valid_0's l2: 0.115628
[343]	valid_0's l2: 0.115643
[344]	valid_0's l2: 0.115552
[345]	valid_0's l2: 0.115512
[346]	valid_0's l2: 0.115459
[347]	valid_0's l2: 0.115434
[348]	valid_0's l2: 0.115415
[349]	valid_0's l2: 0.115389
[350]	valid_0's l2: 0.115343
[351]	valid_0's l2: 0.115299
[352]	valid_0's l2: 0.11531
[353]	valid_0's l2: 0.115358
[354]	valid_0's l2: 0.115296
[355]	valid_0's l2: 0.115279
[356]	valid_0's l2: 0.115275
[357]	valid_0's l2: 0.115236
[358]	valid_0's l2: 0.115306
[359]	valid_0's l2: 0.115319
[360]	valid_0's l2: 0.115298
[361]	valid_0's l2: 0.115298
[362]	valid_0's l2: 0.115309
[363]	valid_0's l2: 0.115352
[364]	valid_0's l2: 0.115392
[365]	valid_0's l2: 0.11543
[366]	valid_0's l2: 0.115422
[367]	valid_0's l2: 0.115375
Early stopping, best iteration is:
[357]	valid_0's l2: 0.115236
lgb now score is: [0.11464525315121991, 0.1001602219958722, 0.10375315940652878, 0.11523585238782036]
[LightGBM] [Warning] Unknown parameter: tree_method
[LightGBM] [Warning] Unknown parameter: colsample_bylevel
[LightGBM] [Warning] Unknown parameter: silent
[LightGBM] [Warning] Unknown parameter: tree_method
[LightGBM] [Warning] Unknown parameter: colsample_bylevel
[LightGBM] [Warning] Unknown parameter: silent
[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.000791 seconds.
You can set `force_col_wise=true` to remove the overhead.
[LightGBM] [Info] Total Bins 8848
[LightGBM] [Info] Number of data points in the train set: 2311, number of used features: 38
[LightGBM] [Warning] Unknown parameter: tree_method
[LightGBM] [Warning] Unknown parameter: colsample_bylevel
[LightGBM] [Warning] Unknown parameter: silent
[LightGBM] [Info] Start training from score 0.130812
[1]	valid_0's l2: 0.936206
Training until validation scores don't improve for 10 rounds
[2]	valid_0's l2: 0.895841
[3]	valid_0's l2: 0.857167
[4]	valid_0's l2: 0.820518
[5]	valid_0's l2: 0.786159
[6]	valid_0's l2: 0.752506
[7]	valid_0's l2: 0.721687
[8]	valid_0's l2: 0.692328
[9]	valid_0's l2: 0.664303
[10]	valid_0's l2: 0.638598
[11]	valid_0's l2: 0.613257
[12]	valid_0's l2: 0.589258
[13]	valid_0's l2: 0.56658
[14]	valid_0's l2: 0.545071
[15]	valid_0's l2: 0.524812
[16]	valid_0's l2: 0.505232
[17]	valid_0's l2: 0.486845
[18]	valid_0's l2: 0.470246
[19]	valid_0's l2: 0.453394
[20]	valid_0's l2: 0.437888
[21]	valid_0's l2: 0.422856
[22]	valid_0's l2: 0.408857
[23]	valid_0's l2: 0.395189
[24]	valid_0's l2: 0.382459
[25]	valid_0's l2: 0.370937
[26]	valid_0's l2: 0.35901
[27]	valid_0's l2: 0.347942
[28]	valid_0's l2: 0.337296
[29]	valid_0's l2: 0.327316
[30]	valid_0's l2: 0.318055
[31]	valid_0's l2: 0.308764
[32]	valid_0's l2: 0.300115
[33]	valid_0's l2: 0.29191
[34]	valid_0's l2: 0.284227
[35]	valid_0's l2: 0.276487
[36]	valid_0's l2: 0.269354
[37]	valid_0's l2: 0.262784
[38]	valid_0's l2: 0.256853
[39]	valid_0's l2: 0.250473
[40]	valid_0's l2: 0.244643
[41]	valid_0's l2: 0.239244
[42]	valid_0's l2: 0.234111
[43]	valid_0's l2: 0.229482
[44]	valid_0's l2: 0.224734
[45]	valid_0's l2: 0.220344
[46]	valid_0's l2: 0.216184
[47]	valid_0's l2: 0.212055
[48]	valid_0's l2: 0.207895
[49]	valid_0's l2: 0.204274
[50]	valid_0's l2: 0.200657
[51]	valid_0's l2: 0.197318
[52]	valid_0's l2: 0.194157
[53]	valid_0's l2: 0.190906
[54]	valid_0's l2: 0.187951
[55]	valid_0's l2: 0.184928
[56]	valid_0's l2: 0.18224
[57]	valid_0's l2: 0.179823
[58]	valid_0's l2: 0.177306
[59]	valid_0's l2: 0.17482
[60]	valid_0's l2: 0.17251
[61]	valid_0's l2: 0.170311
[62]	valid_0's l2: 0.168198
[63]	valid_0's l2: 0.166429
[64]	valid_0's l2: 0.16444
[65]	valid_0's l2: 0.162641
[66]	valid_0's l2: 0.16089
[67]	valid_0's l2: 0.15942
[68]	valid_0's l2: 0.157732
[69]	valid_0's l2: 0.156235
[70]	valid_0's l2: 0.154809
[71]	valid_0's l2: 0.153308
[72]	valid_0's l2: 0.151879
[73]	valid_0's l2: 0.150654
[74]	valid_0's l2: 0.149304
[75]	valid_0's l2: 0.148061
[76]	valid_0's l2: 0.146904
[77]	valid_0's l2: 0.145824
[78]	valid_0's l2: 0.144673
[79]	valid_0's l2: 0.143632
[80]	valid_0's l2: 0.142637
[81]	valid_0's l2: 0.141813
[82]	valid_0's l2: 0.140874
[83]	valid_0's l2: 0.140128
[84]	valid_0's l2: 0.139218
[85]	valid_0's l2: 0.13836
[86]	valid_0's l2: 0.137665
[87]	valid_0's l2: 0.136774
[88]	valid_0's l2: 0.136091
[89]	valid_0's l2: 0.135301
[90]	valid_0's l2: 0.134483
[91]	valid_0's l2: 0.133781
[92]	valid_0's l2: 0.133191
[93]	valid_0's l2: 0.132801
[94]	valid_0's l2: 0.132071
[95]	valid_0's l2: 0.131386
[96]	valid_0's l2: 0.13081
[97]	valid_0's l2: 0.130295
[98]	valid_0's l2: 0.129822
[99]	valid_0's l2: 0.129201
[100]	valid_0's l2: 0.128688
[101]	valid_0's l2: 0.128175
[102]	valid_0's l2: 0.127658
[103]	valid_0's l2: 0.127142
[104]	valid_0's l2: 0.126764
[105]	valid_0's l2: 0.12633
[106]	valid_0's l2: 0.1259
[107]	valid_0's l2: 0.125436
[108]	valid_0's l2: 0.125023
[109]	valid_0's l2: 0.124531
[110]	valid_0's l2: 0.124266
[111]	valid_0's l2: 0.124039
[112]	valid_0's l2: 0.123578
[113]	valid_0's l2: 0.12328
[114]	valid_0's l2: 0.122965
[115]	valid_0's l2: 0.122615
[116]	valid_0's l2: 0.12231
[117]	valid_0's l2: 0.121995
[118]	valid_0's l2: 0.12178
[119]	valid_0's l2: 0.121599
[120]	valid_0's l2: 0.121311
[121]	valid_0's l2: 0.121056
[122]	valid_0's l2: 0.120772
[123]	valid_0's l2: 0.120369
[124]	valid_0's l2: 0.120029
[125]	valid_0's l2: 0.119837
[126]	valid_0's l2: 0.119627
[127]	valid_0's l2: 0.119299
[128]	valid_0's l2: 0.119159
[129]	valid_0's l2: 0.118872
[130]	valid_0's l2: 0.118819
[131]	valid_0's l2: 0.118669
[132]	valid_0's l2: 0.118404
[133]	valid_0's l2: 0.118245
[134]	valid_0's l2: 0.118092
[135]	valid_0's l2: 0.117967
[136]	valid_0's l2: 0.117839
[137]	valid_0's l2: 0.117458
[138]	valid_0's l2: 0.11716
[139]	valid_0's l2: 0.116996
[140]	valid_0's l2: 0.11679
[141]	valid_0's l2: 0.116548
[142]	valid_0's l2: 0.116298
[143]	valid_0's l2: 0.116123
[144]	valid_0's l2: 0.116018
[145]	valid_0's l2: 0.115965
[146]	valid_0's l2: 0.115854
[147]	valid_0's l2: 0.115679
[148]	valid_0's l2: 0.115415
[149]	valid_0's l2: 0.115337
[150]	valid_0's l2: 0.11512
[151]	valid_0's l2: 0.115015
[152]	valid_0's l2: 0.114857
[153]	valid_0's l2: 0.114625
[154]	valid_0's l2: 0.114484
[155]	valid_0's l2: 0.114351
[156]	valid_0's l2: 0.114251
[157]	valid_0's l2: 0.114188
[158]	valid_0's l2: 0.113966
[159]	valid_0's l2: 0.113816
[160]	valid_0's l2: 0.113629
[161]	valid_0's l2: 0.113567
[162]	valid_0's l2: 0.113474
[163]	valid_0's l2: 0.113372
[164]	valid_0's l2: 0.113188
[165]	valid_0's l2: 0.113099
[166]	valid_0's l2: 0.11293
[167]	valid_0's l2: 0.112879
[168]	valid_0's l2: 0.112909
[169]	valid_0's l2: 0.112728
[170]	valid_0's l2: 0.112666
[171]	valid_0's l2: 0.112566
[172]	valid_0's l2: 0.11255
[173]	valid_0's l2: 0.112469
[174]	valid_0's l2: 0.112368
[175]	valid_0's l2: 0.112383
[176]	valid_0's l2: 0.112212
[177]	valid_0's l2: 0.112214
[178]	valid_0's l2: 0.112292
[179]	valid_0's l2: 0.11215
[180]	valid_0's l2: 0.112031
[181]	valid_0's l2: 0.111966
[182]	valid_0's l2: 0.112036
[183]	valid_0's l2: 0.111839
[184]	valid_0's l2: 0.111743
[185]	valid_0's l2: 0.11162
[186]	valid_0's l2: 0.11155
[187]	valid_0's l2: 0.11144
[188]	valid_0's l2: 0.111387
[189]	valid_0's l2: 0.11135
[190]	valid_0's l2: 0.111384
[191]	valid_0's l2: 0.111317
[192]	valid_0's l2: 0.111246
[193]	valid_0's l2: 0.111207
[194]	valid_0's l2: 0.111208
[195]	valid_0's l2: 0.111178
[196]	valid_0's l2: 0.111088
[197]	valid_0's l2: 0.110978
[198]	valid_0's l2: 0.110861
[199]	valid_0's l2: 0.110857
[200]	valid_0's l2: 0.110852
[201]	valid_0's l2: 0.110765
[202]	valid_0's l2: 0.110762
[203]	valid_0's l2: 0.11076
[204]	valid_0's l2: 0.110639
[205]	valid_0's l2: 0.110585
[206]	valid_0's l2: 0.110508
[207]	valid_0's l2: 0.110447
[208]	valid_0's l2: 0.110415
[209]	valid_0's l2: 0.110324
[210]	valid_0's l2: 0.110356
[211]	valid_0's l2: 0.110303
[212]	valid_0's l2: 0.110307
[213]	valid_0's l2: 0.110288
[214]	valid_0's l2: 0.110188
[215]	valid_0's l2: 0.110091
[216]	valid_0's l2: 0.110052
[217]	valid_0's l2: 0.109953
[218]	valid_0's l2: 0.109941
[219]	valid_0's l2: 0.109894
[220]	valid_0's l2: 0.109813
[221]	valid_0's l2: 0.10971
[222]	valid_0's l2: 0.10966
[223]	valid_0's l2: 0.109711
[224]	valid_0's l2: 0.109603
[225]	valid_0's l2: 0.109622
[226]	valid_0's l2: 0.109553
[227]	valid_0's l2: 0.10954
[228]	valid_0's l2: 0.109503
[229]	valid_0's l2: 0.1094
[230]	valid_0's l2: 0.109398
[231]	valid_0's l2: 0.109377
[232]	valid_0's l2: 0.109411
[233]	valid_0's l2: 0.109335
[234]	valid_0's l2: 0.109301
[235]	valid_0's l2: 0.109261
[236]	valid_0's l2: 0.109191
[237]	valid_0's l2: 0.109043
[238]	valid_0's l2: 0.10901
[239]	valid_0's l2: 0.108991
[240]	valid_0's l2: 0.109039
[241]	valid_0's l2: 0.108958
[242]	valid_0's l2: 0.108935
[243]	valid_0's l2: 0.10891
[244]	valid_0's l2: 0.108871
[245]	valid_0's l2: 0.108813
[246]	valid_0's l2: 0.108771
[247]	valid_0's l2: 0.108763
[248]	valid_0's l2: 0.108667
[249]	valid_0's l2: 0.108628
[250]	valid_0's l2: 0.108674
[251]	valid_0's l2: 0.10863
[252]	valid_0's l2: 0.108624
[253]	valid_0's l2: 0.108566
[254]	valid_0's l2: 0.108488
[255]	valid_0's l2: 0.108443
[256]	valid_0's l2: 0.108413
[257]	valid_0's l2: 0.108392
[258]	valid_0's l2: 0.108343
[259]	valid_0's l2: 0.10836
[260]	valid_0's l2: 0.108346
[261]	valid_0's l2: 0.108361
[262]	valid_0's l2: 0.108367
[263]	valid_0's l2: 0.108289
[264]	valid_0's l2: 0.108259
[265]	valid_0's l2: 0.108212
[266]	valid_0's l2: 0.108262
[267]	valid_0's l2: 0.10822
[268]	valid_0's l2: 0.108101
[269]	valid_0's l2: 0.108019
[270]	valid_0's l2: 0.108008
[271]	valid_0's l2: 0.107976
[272]	valid_0's l2: 0.108033
[273]	valid_0's l2: 0.107989
[274]	valid_0's l2: 0.107926
[275]	valid_0's l2: 0.107947
[276]	valid_0's l2: 0.107964
[277]	valid_0's l2: 0.107889
[278]	valid_0's l2: 0.107856
[279]	valid_0's l2: 0.107835
[280]	valid_0's l2: 0.107718
[281]	valid_0's l2: 0.107649
[282]	valid_0's l2: 0.107641
[283]	valid_0's l2: 0.107624
[284]	valid_0's l2: 0.107612
[285]	valid_0's l2: 0.1076
[286]	valid_0's l2: 0.10755
[287]	valid_0's l2: 0.107525
[288]	valid_0's l2: 0.107403
[289]	valid_0's l2: 0.107386
[290]	valid_0's l2: 0.107421
[291]	valid_0's l2: 0.107425
[292]	valid_0's l2: 0.107383
[293]	valid_0's l2: 0.107328
[294]	valid_0's l2: 0.107313
[295]	valid_0's l2: 0.107297
[296]	valid_0's l2: 0.107275
[297]	valid_0's l2: 0.107286
[298]	valid_0's l2: 0.107273
[299]	valid_0's l2: 0.107216
[300]	valid_0's l2: 0.107143
[301]	valid_0's l2: 0.107092
[302]	valid_0's l2: 0.107074
[303]	valid_0's l2: 0.107119
[304]	valid_0's l2: 0.107051
[305]	valid_0's l2: 0.107027
[306]	valid_0's l2: 0.106993
[307]	valid_0's l2: 0.106946
[308]	valid_0's l2: 0.106894
[309]	valid_0's l2: 0.10686
[310]	valid_0's l2: 0.106841
[311]	valid_0's l2: 0.106812
[312]	valid_0's l2: 0.106788
[313]	valid_0's l2: 0.106774
[314]	valid_0's l2: 0.106784
[315]	valid_0's l2: 0.106725
[316]	valid_0's l2: 0.106675
[317]	valid_0's l2: 0.106631
[318]	valid_0's l2: 0.106652
[319]	valid_0's l2: 0.106643
[320]	valid_0's l2: 0.106649
[321]	valid_0's l2: 0.106614
[322]	valid_0's l2: 0.106688
[323]	valid_0's l2: 0.106714
[324]	valid_0's l2: 0.106738
[325]	valid_0's l2: 0.106739
[326]	valid_0's l2: 0.106729
[327]	valid_0's l2: 0.10675
[328]	valid_0's l2: 0.106749
[329]	valid_0's l2: 0.106699
[330]	valid_0's l2: 0.106733
[331]	valid_0's l2: 0.106719
Early stopping, best iteration is:
[321]	valid_0's l2: 0.106614
lgb now score is: [0.11464525315121991, 0.1001602219958722, 0.10375315940652878, 0.11523585238782036, 0.10661431579572031]
lgb_score_list: [0.11464525315121991, 0.1001602219958722, 0.10375315940652878, 0.11523585238782036, 0.10661431579572031]
lgb_score_mean: 0.1080817605474323
(2888, 38)
(2888, 39)
lgb_reg
False
lgb_reg
[LightGBM] [Warning] Unknown parameter: tree_method
[LightGBM] [Warning] Unknown parameter: colsample_bylevel
[LightGBM] [Warning] Unknown parameter: silent
[LightGBM] [Warning] Unknown parameter: tree_method
[LightGBM] [Warning] Unknown parameter: colsample_bylevel
[LightGBM] [Warning] Unknown parameter: silent
[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.000872 seconds.
You can set `force_col_wise=true` to remove the overhead.
[LightGBM] [Info] Total Bins 9126
[LightGBM] [Info] Number of data points in the train set: 2888, number of used features: 39
[LightGBM] [Warning] Unknown parameter: tree_method
[LightGBM] [Warning] Unknown parameter: colsample_bylevel
[LightGBM] [Warning] Unknown parameter: silent
[LightGBM] [Info] Start training from score 0.126353
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[1]	valid_0's l2: 0.920198
Training until validation scores don't improve for 10 rounds
[2]	valid_0's l2: 0.877501
[3]	valid_0's l2: 0.837096
[4]	valid_0's l2: 0.796634
[5]	valid_0's l2: 0.760988
[6]	valid_0's l2: 0.726895
[7]	valid_0's l2: 0.692514
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[8]	valid_0's l2: 0.66003
[9]	valid_0's l2: 0.629321
[10]	valid_0's l2: 0.600303
[11]	valid_0's l2: 0.574484
[12]	valid_0's l2: 0.550053
[13]	valid_0's l2: 0.525398
[14]	valid_0's l2: 0.503693
[15]	valid_0's l2: 0.481571
[16]	valid_0's l2: 0.460633
[17]	valid_0's l2: 0.440842
[18]	valid_0's l2: 0.422154
[19]	valid_0's l2: 0.405527
[20]	valid_0's l2: 0.388729
[21]	valid_0's l2: 0.372852
[22]	valid_0's l2: 0.358731
[23]	valid_0's l2: 0.344477
[24]	valid_0's l2: 0.331865
[25]	valid_0's l2: 0.320117
[26]	valid_0's l2: 0.30877
[27]	valid_0's l2: 0.297215
[28]	valid_0's l2: 0.286285
[29]	valid_0's l2: 0.276795
[30]	valid_0's l2: 0.266987
[31]	valid_0's l2: 0.257675
[32]	valid_0's l2: 0.248847
[33]	valid_0's l2: 0.240504
[34]	valid_0's l2: 0.23261
[35]	valid_0's l2: 0.225159
[36]	valid_0's l2: 0.21812
[37]	valid_0's l2: 0.211375
[38]	valid_0's l2: 0.205017
[39]	valid_0's l2: 0.199026
[40]	valid_0's l2: 0.193628
[41]	valid_0's l2: 0.188137
[42]	valid_0's l2: 0.183008
[43]	valid_0's l2: 0.178142
[44]	valid_0's l2: 0.173445
[45]	valid_0's l2: 0.169077
[46]	valid_0's l2: 0.165068
[47]	valid_0's l2: 0.161044
[48]	valid_0's l2: 0.157256
[49]	valid_0's l2: 0.153645
[50]	valid_0's l2: 0.150257
[51]	valid_0's l2: 0.146919
[52]	valid_0's l2: 0.143854
[53]	valid_0's l2: 0.140868
[54]	valid_0's l2: 0.137998
[55]	valid_0's l2: 0.135382
[56]	valid_0's l2: 0.132761
[57]	valid_0's l2: 0.130272
[58]	valid_0's l2: 0.12792
[59]	valid_0's l2: 0.125694
[60]	valid_0's l2: 0.123562
[61]	valid_0's l2: 0.12158
[62]	valid_0's l2: 0.119611
[63]	valid_0's l2: 0.117707
[64]	valid_0's l2: 0.115863
[65]	valid_0's l2: 0.114163
[66]	valid_0's l2: 0.112492
[67]	valid_0's l2: 0.110916
[68]	valid_0's l2: 0.109392
[69]	valid_0's l2: 0.107826
[70]	valid_0's l2: 0.106362
[71]	valid_0's l2: 0.10497
[72]	valid_0's l2: 0.103623
[73]	valid_0's l2: 0.10234
[74]	valid_0's l2: 0.101078
[75]	valid_0's l2: 0.0998728
[76]	valid_0's l2: 0.0987565
[77]	valid_0's l2: 0.0976769
[78]	valid_0's l2: 0.0966
[79]	valid_0's l2: 0.0956068
[80]	valid_0's l2: 0.0946104
[81]	valid_0's l2: 0.093656
[82]	valid_0's l2: 0.0926436
[83]	valid_0's l2: 0.0917123
[84]	valid_0's l2: 0.090806
[85]	valid_0's l2: 0.0899506
[86]	valid_0's l2: 0.0891238
[87]	valid_0's l2: 0.0883361
[88]	valid_0's l2: 0.0875694
[89]	valid_0's l2: 0.0868473
[90]	valid_0's l2: 0.0860895
[91]	valid_0's l2: 0.0853649
[92]	valid_0's l2: 0.0846328
[93]	valid_0's l2: 0.0838801
[94]	valid_0's l2: 0.0832734
[95]	valid_0's l2: 0.082608
[96]	valid_0's l2: 0.0819408
[97]	valid_0's l2: 0.0813099
[98]	valid_0's l2: 0.0807066
[99]	valid_0's l2: 0.0800998
[100]	valid_0's l2: 0.079549
[101]	valid_0's l2: 0.0789841
[102]	valid_0's l2: 0.0784413
[103]	valid_0's l2: 0.0778946
[104]	valid_0's l2: 0.0773375
[105]	valid_0's l2: 0.076757
[106]	valid_0's l2: 0.076234
[107]	valid_0's l2: 0.0757108
[108]	valid_0's l2: 0.0752375
[109]	valid_0's l2: 0.0747273
[110]	valid_0's l2: 0.0742649
[111]	valid_0's l2: 0.0737894
[112]	valid_0's l2: 0.0733377
[113]	valid_0's l2: 0.0728907
[114]	valid_0's l2: 0.0724813
[115]	valid_0's l2: 0.0720514
[116]	valid_0's l2: 0.0716235
[117]	valid_0's l2: 0.0712518
[118]	valid_0's l2: 0.0708029
[119]	valid_0's l2: 0.0703807
[120]	valid_0's l2: 0.0699569
[121]	valid_0's l2: 0.0695757
[122]	valid_0's l2: 0.069188
[123]	valid_0's l2: 0.0688301
[124]	valid_0's l2: 0.0684313
[125]	valid_0's l2: 0.0680283
[126]	valid_0's l2: 0.0676548
[127]	valid_0's l2: 0.0672451
[128]	valid_0's l2: 0.0668647
[129]	valid_0's l2: 0.0664871
[130]	valid_0's l2: 0.0661413
[131]	valid_0's l2: 0.065796
[132]	valid_0's l2: 0.0654315
[133]	valid_0's l2: 0.0650831
[134]	valid_0's l2: 0.0647244
[135]	valid_0's l2: 0.0643841
[136]	valid_0's l2: 0.0640391
[137]	valid_0's l2: 0.0637384
[138]	valid_0's l2: 0.0634262
[139]	valid_0's l2: 0.0631044
[140]	valid_0's l2: 0.062778
[141]	valid_0's l2: 0.0624782
[142]	valid_0's l2: 0.0621561
[143]	valid_0's l2: 0.0618215
[144]	valid_0's l2: 0.0615053
[145]	valid_0's l2: 0.0611636
[146]	valid_0's l2: 0.0608256
[147]	valid_0's l2: 0.0605239
[148]	valid_0's l2: 0.0602332
[149]	valid_0's l2: 0.05994
[150]	valid_0's l2: 0.0596534
[151]	valid_0's l2: 0.0593666
[152]	valid_0's l2: 0.0590838
[153]	valid_0's l2: 0.0588218
[154]	valid_0's l2: 0.0585661
[155]	valid_0's l2: 0.0582825
[156]	valid_0's l2: 0.0580247
[157]	valid_0's l2: 0.0577481
[158]	valid_0's l2: 0.0574347
[159]	valid_0's l2: 0.0571654
[160]	valid_0's l2: 0.0568895
[161]	valid_0's l2: 0.0566506
[162]	valid_0's l2: 0.05642
[163]	valid_0's l2: 0.0561828
[164]	valid_0's l2: 0.0559371
[165]	valid_0's l2: 0.0557143
[166]	valid_0's l2: 0.0554341
[167]	valid_0's l2: 0.0552245
[168]	valid_0's l2: 0.0549674
[169]	valid_0's l2: 0.0547357
[170]	valid_0's l2: 0.0544524
[171]	valid_0's l2: 0.0541773
[172]	valid_0's l2: 0.0539216
[173]	valid_0's l2: 0.0536876
[174]	valid_0's l2: 0.0534448
[175]	valid_0's l2: 0.0532163
[176]	valid_0's l2: 0.0529931
[177]	valid_0's l2: 0.0527497
[178]	valid_0's l2: 0.0525413
[179]	valid_0's l2: 0.0523342
[180]	valid_0's l2: 0.0520939
[181]	valid_0's l2: 0.0518482
[182]	valid_0's l2: 0.0515886
[183]	valid_0's l2: 0.0513792
[184]	valid_0's l2: 0.0511629
[185]	valid_0's l2: 0.050926
[186]	valid_0's l2: 0.0507362
[187]	valid_0's l2: 0.0505168
[188]	valid_0's l2: 0.0503015
[189]	valid_0's l2: 0.0501021
[190]	valid_0's l2: 0.0498635
[191]	valid_0's l2: 0.0496362
[192]	valid_0's l2: 0.0494557
[193]	valid_0's l2: 0.0492321
[194]	valid_0's l2: 0.0490165
[195]	valid_0's l2: 0.0488094
[196]	valid_0's l2: 0.0486346
[197]	valid_0's l2: 0.0484021
[198]	valid_0's l2: 0.0482034
[199]	valid_0's l2: 0.0480083
[200]	valid_0's l2: 0.0478169
[201]	valid_0's l2: 0.047626
[202]	valid_0's l2: 0.0474026
[203]	valid_0's l2: 0.0472183
[204]	valid_0's l2: 0.046995
[205]	valid_0's l2: 0.0467757
[206]	valid_0's l2: 0.0465913
[207]	valid_0's l2: 0.0463583
[208]	valid_0's l2: 0.04615
[209]	valid_0's l2: 0.0459667
[210]	valid_0's l2: 0.0457877
[211]	valid_0's l2: 0.0455888
[212]	valid_0's l2: 0.0454067
[213]	valid_0's l2: 0.0452036
[214]	valid_0's l2: 0.0450257
[215]	valid_0's l2: 0.0448604
[216]	valid_0's l2: 0.044662
[217]	valid_0's l2: 0.0444738
[218]	valid_0's l2: 0.0443127
[219]	valid_0's l2: 0.0441101
[220]	valid_0's l2: 0.0439273
[221]	valid_0's l2: 0.043749
[222]	valid_0's l2: 0.0435535
[223]	valid_0's l2: 0.0434021
[224]	valid_0's l2: 0.0432227
[225]	valid_0's l2: 0.043051
[226]	valid_0's l2: 0.0428982
[227]	valid_0's l2: 0.0427408
[228]	valid_0's l2: 0.0425735
[229]	valid_0's l2: 0.0424031
[230]	valid_0's l2: 0.0422558
[231]	valid_0's l2: 0.0420839
[232]	valid_0's l2: 0.0419328
[233]	valid_0's l2: 0.0417914
[234]	valid_0's l2: 0.041622
[235]	valid_0's l2: 0.0414701
[236]	valid_0's l2: 0.0412914
[237]	valid_0's l2: 0.0411226
[238]	valid_0's l2: 0.0409581
[239]	valid_0's l2: 0.0407911
[240]	valid_0's l2: 0.0406334
[241]	valid_0's l2: 0.0404966
[242]	valid_0's l2: 0.0403621
[243]	valid_0's l2: 0.0402119
[244]	valid_0's l2: 0.0400429
[245]	valid_0's l2: 0.039876
[246]	valid_0's l2: 0.0397355
[247]	valid_0's l2: 0.0395668
[248]	valid_0's l2: 0.0394274
[249]	valid_0's l2: 0.0392655
[250]	valid_0's l2: 0.0391132
[251]	valid_0's l2: 0.0389681
[252]	valid_0's l2: 0.0388175
[253]	valid_0's l2: 0.0386789
[254]	valid_0's l2: 0.0385332
[255]	valid_0's l2: 0.0383852
[256]	valid_0's l2: 0.0382532
[257]	valid_0's l2: 0.0380884
[258]	valid_0's l2: 0.037932
[259]	valid_0's l2: 0.0377994
[260]	valid_0's l2: 0.0376666
[261]	valid_0's l2: 0.0375289
[262]	valid_0's l2: 0.0373884
[263]	valid_0's l2: 0.0372284
[264]	valid_0's l2: 0.0370883
[265]	valid_0's l2: 0.0369467
[266]	valid_0's l2: 0.0367985
[267]	valid_0's l2: 0.0366672
[268]	valid_0's l2: 0.0365261
[269]	valid_0's l2: 0.0364001
[270]	valid_0's l2: 0.0362617
[271]	valid_0's l2: 0.03614
[272]	valid_0's l2: 0.0359984
[273]	valid_0's l2: 0.0358566
[274]	valid_0's l2: 0.0357353
[275]	valid_0's l2: 0.0356
[276]	valid_0's l2: 0.0354529
[277]	valid_0's l2: 0.0353143
[278]	valid_0's l2: 0.0351655
[279]	valid_0's l2: 0.0350389
[280]	valid_0's l2: 0.0349027
[281]	valid_0's l2: 0.0347735
[282]	valid_0's l2: 0.034644
[283]	valid_0's l2: 0.0345016
[284]	valid_0's l2: 0.0343841
[285]	valid_0's l2: 0.0342532
[286]	valid_0's l2: 0.0341224
[287]	valid_0's l2: 0.0339967
[288]	valid_0's l2: 0.0338592
[289]	valid_0's l2: 0.0337322
[290]	valid_0's l2: 0.0336135
[291]	valid_0's l2: 0.0334987
[292]	valid_0's l2: 0.0333646
[293]	valid_0's l2: 0.0332563
[294]	valid_0's l2: 0.0331167
[295]	valid_0's l2: 0.0330035
[296]	valid_0's l2: 0.0328742
[297]	valid_0's l2: 0.0327561
[298]	valid_0's l2: 0.0326382
[299]	valid_0's l2: 0.0325294
[300]	valid_0's l2: 0.0324194
[301]	valid_0's l2: 0.0323175
[302]	valid_0's l2: 0.0321924
[303]	valid_0's l2: 0.0320636
[304]	valid_0's l2: 0.0319467
[305]	valid_0's l2: 0.0317993
[306]	valid_0's l2: 0.0316835
[307]	valid_0's l2: 0.0315738
[308]	valid_0's l2: 0.0314605
[309]	valid_0's l2: 0.0313546
[310]	valid_0's l2: 0.0312225
[311]	valid_0's l2: 0.0311004
[312]	valid_0's l2: 0.0309879
[313]	valid_0's l2: 0.0308794
[314]	valid_0's l2: 0.0307566
[315]	valid_0's l2: 0.0306497
[316]	valid_0's l2: 0.0305359
[317]	valid_0's l2: 0.0304253
[318]	valid_0's l2: 0.0303187
[319]	valid_0's l2: 0.0302078
[320]	valid_0's l2: 0.0300841
[321]	valid_0's l2: 0.0299782
[322]	valid_0's l2: 0.0298677
[323]	valid_0's l2: 0.0297692
[324]	valid_0's l2: 0.0296543
[325]	valid_0's l2: 0.0295466
[326]	valid_0's l2: 0.0294533
[327]	valid_0's l2: 0.0293471
[328]	valid_0's l2: 0.0292263
[329]	valid_0's l2: 0.0291237
[330]	valid_0's l2: 0.029026
[331]	valid_0's l2: 0.028925
[332]	valid_0's l2: 0.0288194
[333]	valid_0's l2: 0.0287153
[334]	valid_0's l2: 0.0286177
[335]	valid_0's l2: 0.0285273
[336]	valid_0's l2: 0.0284344
[337]	valid_0's l2: 0.028326
[338]	valid_0's l2: 0.0282405
[339]	valid_0's l2: 0.0281493
[340]	valid_0's l2: 0.0280444
[341]	valid_0's l2: 0.0279395
[342]	valid_0's l2: 0.0278497
[343]	valid_0's l2: 0.0277544
[344]	valid_0's l2: 0.0276564
[345]	valid_0's l2: 0.027556
[346]	valid_0's l2: 0.0274631
[347]	valid_0's l2: 0.0273584
[348]	valid_0's l2: 0.0272571
[349]	valid_0's l2: 0.0271598
[350]	valid_0's l2: 0.0270661
[351]	valid_0's l2: 0.0269656
[352]	valid_0's l2: 0.026872
[353]	valid_0's l2: 0.0267654
[354]	valid_0's l2: 0.026683
[355]	valid_0's l2: 0.0265876
[356]	valid_0's l2: 0.0265064
[357]	valid_0's l2: 0.0264206
[358]	valid_0's l2: 0.0263261
[359]	valid_0's l2: 0.0262268
[360]	valid_0's l2: 0.0261385
[361]	valid_0's l2: 0.0260491
[362]	valid_0's l2: 0.0259586
[363]	valid_0's l2: 0.0258718
[364]	valid_0's l2: 0.0257845
[365]	valid_0's l2: 0.025698
[366]	valid_0's l2: 0.0256091
[367]	valid_0's l2: 0.0255066
[368]	valid_0's l2: 0.025414
[369]	valid_0's l2: 0.0253178
[370]	valid_0's l2: 0.0252268
[371]	valid_0's l2: 0.0251415
[372]	valid_0's l2: 0.0250536
[373]	valid_0's l2: 0.0249656
[374]	valid_0's l2: 0.0248922
[375]	valid_0's l2: 0.0248076
[376]	valid_0's l2: 0.0247133
[377]	valid_0's l2: 0.0246336
[378]	valid_0's l2: 0.0245543
[379]	valid_0's l2: 0.024463
[380]	valid_0's l2: 0.0243908
[381]	valid_0's l2: 0.0243089
[382]	valid_0's l2: 0.0242223
[383]	valid_0's l2: 0.0241504
[384]	valid_0's l2: 0.0240773
[385]	valid_0's l2: 0.0239832
[386]	valid_0's l2: 0.0238987
[387]	valid_0's l2: 0.0238124
[388]	valid_0's l2: 0.0237279
[389]	valid_0's l2: 0.0236464
[390]	valid_0's l2: 0.0235599
[391]	valid_0's l2: 0.0234739
[392]	valid_0's l2: 0.0233941
[393]	valid_0's l2: 0.0233051
[394]	valid_0's l2: 0.0232125
[395]	valid_0's l2: 0.0231359
[396]	valid_0's l2: 0.0230624
[397]	valid_0's l2: 0.0229959
[398]	valid_0's l2: 0.0229201
[399]	valid_0's l2: 0.0228434
[400]	valid_0's l2: 0.0227616
[401]	valid_0's l2: 0.0226815
[402]	valid_0's l2: 0.022607
[403]	valid_0's l2: 0.0225342
[404]	valid_0's l2: 0.0224529
[405]	valid_0's l2: 0.0223827
[406]	valid_0's l2: 0.0223098
[407]	valid_0's l2: 0.022241
[408]	valid_0's l2: 0.0221678
[409]	valid_0's l2: 0.0220848
[410]	valid_0's l2: 0.0220053
[411]	valid_0's l2: 0.0219357
[412]	valid_0's l2: 0.0218662
[413]	valid_0's l2: 0.0217852
[414]	valid_0's l2: 0.0217066
[415]	valid_0's l2: 0.0216262
[416]	valid_0's l2: 0.0215557
[417]	valid_0's l2: 0.0214884
[418]	valid_0's l2: 0.0214134
[419]	valid_0's l2: 0.0213336
[420]	valid_0's l2: 0.0212735
[421]	valid_0's l2: 0.0212104
[422]	valid_0's l2: 0.0211432
[423]	valid_0's l2: 0.021079
[424]	valid_0's l2: 0.0210032
[425]	valid_0's l2: 0.0209392
[426]	valid_0's l2: 0.0208721
[427]	valid_0's l2: 0.0208122
[428]	valid_0's l2: 0.0207427
[429]	valid_0's l2: 0.0206784
[430]	valid_0's l2: 0.0205995
[431]	valid_0's l2: 0.0205368
[432]	valid_0's l2: 0.0204694
[433]	valid_0's l2: 0.0204004
[434]	valid_0's l2: 0.0203348
[435]	valid_0's l2: 0.0202709
[436]	valid_0's l2: 0.0202165
[437]	valid_0's l2: 0.0201513
[438]	valid_0's l2: 0.0200816
[439]	valid_0's l2: 0.02002
[440]	valid_0's l2: 0.0199487
[441]	valid_0's l2: 0.019885
[442]	valid_0's l2: 0.0198272
[443]	valid_0's l2: 0.0197579
[444]	valid_0's l2: 0.0196982
[445]	valid_0's l2: 0.0196292
[446]	valid_0's l2: 0.0195708
[447]	valid_0's l2: 0.0195081
[448]	valid_0's l2: 0.0194422
[449]	valid_0's l2: 0.0193768
[450]	valid_0's l2: 0.0193136
[451]	valid_0's l2: 0.0192471
[452]	valid_0's l2: 0.0191747
[453]	valid_0's l2: 0.0191124
[454]	valid_0's l2: 0.0190483
[455]	valid_0's l2: 0.0189932
[456]	valid_0's l2: 0.0189274
[457]	valid_0's l2: 0.0188666
[458]	valid_0's l2: 0.0188057
[459]	valid_0's l2: 0.0187468
[460]	valid_0's l2: 0.0186874
[461]	valid_0's l2: 0.0186373
[462]	valid_0's l2: 0.0185759
[463]	valid_0's l2: 0.0185127
[464]	valid_0's l2: 0.0184518
[465]	valid_0's l2: 0.0183863
[466]	valid_0's l2: 0.0183335
[467]	valid_0's l2: 0.0182767
[468]	valid_0's l2: 0.0182227
[469]	valid_0's l2: 0.0181616
[470]	valid_0's l2: 0.0181034
[471]	valid_0's l2: 0.0180449
[472]	valid_0's l2: 0.0179857
[473]	valid_0's l2: 0.0179243
[474]	valid_0's l2: 0.017864
[475]	valid_0's l2: 0.017811
[476]	valid_0's l2: 0.0177563
[477]	valid_0's l2: 0.0176983
[478]	valid_0's l2: 0.0176437
[479]	valid_0's l2: 0.017594
[480]	valid_0's l2: 0.0175356
[481]	valid_0's l2: 0.0174881
[482]	valid_0's l2: 0.0174288
[483]	valid_0's l2: 0.017371
[484]	valid_0's l2: 0.0173169
[485]	valid_0's l2: 0.0172679
[486]	valid_0's l2: 0.017213
[487]	valid_0's l2: 0.0171669
[488]	valid_0's l2: 0.0171131
[489]	valid_0's l2: 0.017055
[490]	valid_0's l2: 0.0169895
[491]	valid_0's l2: 0.0169398
[492]	valid_0's l2: 0.0168845
[493]	valid_0's l2: 0.0168276
[494]	valid_0's l2: 0.0167689
[495]	valid_0's l2: 0.0167261
[496]	valid_0's l2: 0.0166614
[497]	valid_0's l2: 0.0166035
[498]	valid_0's l2: 0.0165538
[499]	valid_0's l2: 0.0164969
[500]	valid_0's l2: 0.0164417
[501]	valid_0's l2: 0.016396
[502]	valid_0's l2: 0.016347
[503]	valid_0's l2: 0.016293
[504]	valid_0's l2: 0.0162467
[505]	valid_0's l2: 0.0161936
[506]	valid_0's l2: 0.0161354
[507]	valid_0's l2: 0.0160856
[508]	valid_0's l2: 0.0160306
[509]	valid_0's l2: 0.0159787
[510]	valid_0's l2: 0.0159252
[511]	valid_0's l2: 0.015869
[512]	valid_0's l2: 0.0158201
[513]	valid_0's l2: 0.0157738
[514]	valid_0's l2: 0.0157277
[515]	valid_0's l2: 0.0156685
[516]	valid_0's l2: 0.0156203
[517]	valid_0's l2: 0.0155731
[518]	valid_0's l2: 0.0155216
[519]	valid_0's l2: 0.0154731
[520]	valid_0's l2: 0.0154257
[521]	valid_0's l2: 0.0153855
[522]	valid_0's l2: 0.0153433
[523]	valid_0's l2: 0.0152947
[524]	valid_0's l2: 0.0152412
[525]	valid_0's l2: 0.0151917
[526]	valid_0's l2: 0.0151513
[527]	valid_0's l2: 0.0151026
[528]	valid_0's l2: 0.0150631
[529]	valid_0's l2: 0.0150194
[530]	valid_0's l2: 0.0149765
[531]	valid_0's l2: 0.0149336
[532]	valid_0's l2: 0.0148831
[533]	valid_0's l2: 0.014834
[534]	valid_0's l2: 0.0147868
[535]	valid_0's l2: 0.0147363
[536]	valid_0's l2: 0.0146885
[537]	valid_0's l2: 0.014646
[538]	valid_0's l2: 0.0145951
[539]	valid_0's l2: 0.0145546
[540]	valid_0's l2: 0.0145084
[541]	valid_0's l2: 0.0144638
[542]	valid_0's l2: 0.0144179
[543]	valid_0's l2: 0.0143723
[544]	valid_0's l2: 0.0143271
[545]	valid_0's l2: 0.0142815
[546]	valid_0's l2: 0.0142391
[547]	valid_0's l2: 0.0141955
[548]	valid_0's l2: 0.0141514
[549]	valid_0's l2: 0.0141118
[550]	valid_0's l2: 0.0140631
[551]	valid_0's l2: 0.0140253
[552]	valid_0's l2: 0.0139813
[553]	valid_0's l2: 0.0139389
[554]	valid_0's l2: 0.0138994
[555]	valid_0's l2: 0.013859
[556]	valid_0's l2: 0.0138089
[557]	valid_0's l2: 0.0137652
[558]	valid_0's l2: 0.0137244
[559]	valid_0's l2: 0.013682
[560]	valid_0's l2: 0.013642
[561]	valid_0's l2: 0.0135916
[562]	valid_0's l2: 0.0135578
[563]	valid_0's l2: 0.0135169
[564]	valid_0's l2: 0.0134765
[565]	valid_0's l2: 0.013426
[566]	valid_0's l2: 0.0133852
[567]	valid_0's l2: 0.01334
[568]	valid_0's l2: 0.013302
[569]	valid_0's l2: 0.0132602
[570]	valid_0's l2: 0.013221
[571]	valid_0's l2: 0.0131795
[572]	valid_0's l2: 0.0131393
[573]	valid_0's l2: 0.0131015
[574]	valid_0's l2: 0.0130655
[575]	valid_0's l2: 0.0130266
[576]	valid_0's l2: 0.0129882
[577]	valid_0's l2: 0.0129407
[578]	valid_0's l2: 0.0129036
[579]	valid_0's l2: 0.0128704
[580]	valid_0's l2: 0.01283
[581]	valid_0's l2: 0.0127951
[582]	valid_0's l2: 0.0127581
[583]	valid_0's l2: 0.0127225
[584]	valid_0's l2: 0.0126852
[585]	valid_0's l2: 0.0126508
[586]	valid_0's l2: 0.0126157
[587]	valid_0's l2: 0.0125778
[588]	valid_0's l2: 0.0125396
[589]	valid_0's l2: 0.0125093
[590]	valid_0's l2: 0.0124753
[591]	valid_0's l2: 0.0124364
[592]	valid_0's l2: 0.0123992
[593]	valid_0's l2: 0.0123644
[594]	valid_0's l2: 0.012324
[595]	valid_0's l2: 0.0122882
[596]	valid_0's l2: 0.012247
[597]	valid_0's l2: 0.0122074
[598]	valid_0's l2: 0.0121702
[599]	valid_0's l2: 0.0121326
[600]	valid_0's l2: 0.0120962
[601]	valid_0's l2: 0.0120611
[602]	valid_0's l2: 0.0120274
[603]	valid_0's l2: 0.0119917
[604]	valid_0's l2: 0.0119589
[605]	valid_0's l2: 0.0119258
[606]	valid_0's l2: 0.01189
[607]	valid_0's l2: 0.0118473
[608]	valid_0's l2: 0.0118075
[609]	valid_0's l2: 0.0117714
[610]	valid_0's l2: 0.0117349
[611]	valid_0's l2: 0.0117024
[612]	valid_0's l2: 0.0116726
[613]	valid_0's l2: 0.0116377
[614]	valid_0's l2: 0.0116028
[615]	valid_0's l2: 0.011567
[616]	valid_0's l2: 0.011533
[617]	valid_0's l2: 0.011501
[618]	valid_0's l2: 0.0114642
[619]	valid_0's l2: 0.0114297
[620]	valid_0's l2: 0.0113969
[621]	valid_0's l2: 0.011357
[622]	valid_0's l2: 0.0113201
[623]	valid_0's l2: 0.0112868
[624]	valid_0's l2: 0.011254
[625]	valid_0's l2: 0.011224
[626]	valid_0's l2: 0.011191
[627]	valid_0's l2: 0.0111522
[628]	valid_0's l2: 0.011113
[629]	valid_0's l2: 0.0110821
[630]	valid_0's l2: 0.0110482
[631]	valid_0's l2: 0.0110084
[632]	valid_0's l2: 0.0109698
[633]	valid_0's l2: 0.0109339
[634]	valid_0's l2: 0.0109028
[635]	valid_0's l2: 0.0108721
[636]	valid_0's l2: 0.0108381
[637]	valid_0's l2: 0.0108081
[638]	valid_0's l2: 0.0107805
[639]	valid_0's l2: 0.0107546
[640]	valid_0's l2: 0.0107213
[641]	valid_0's l2: 0.0106916
[642]	valid_0's l2: 0.01066
[643]	valid_0's l2: 0.0106281
[644]	valid_0's l2: 0.0105954
[645]	valid_0's l2: 0.0105681
[646]	valid_0's l2: 0.0105409
[647]	valid_0's l2: 0.0105041
[648]	valid_0's l2: 0.0104694
[649]	valid_0's l2: 0.010437
[650]	valid_0's l2: 0.0104062
[651]	valid_0's l2: 0.0103761
[652]	valid_0's l2: 0.0103435
[653]	valid_0's l2: 0.0103075
[654]	valid_0's l2: 0.0102794
[655]	valid_0's l2: 0.0102512
[656]	valid_0's l2: 0.0102219
[657]	valid_0's l2: 0.0101965
[658]	valid_0's l2: 0.0101652
[659]	valid_0's l2: 0.0101386
[660]	valid_0's l2: 0.0101061
[661]	valid_0's l2: 0.0100746
[662]	valid_0's l2: 0.0100489
[663]	valid_0's l2: 0.0100162
[664]	valid_0's l2: 0.00998765
[665]	valid_0's l2: 0.00996005
[666]	valid_0's l2: 0.00993276
[667]	valid_0's l2: 0.00990016
[668]	valid_0's l2: 0.00987478
[669]	valid_0's l2: 0.00984551
[670]	valid_0's l2: 0.00981317
[671]	valid_0's l2: 0.0097865
[672]	valid_0's l2: 0.00975501
[673]	valid_0's l2: 0.00972338
[674]	valid_0's l2: 0.00969267
[675]	valid_0's l2: 0.00966437
[676]	valid_0's l2: 0.00964019
[677]	valid_0's l2: 0.00961377
[678]	valid_0's l2: 0.00958592
[679]	valid_0's l2: 0.00955535
[680]	valid_0's l2: 0.00953371
[681]	valid_0's l2: 0.00950365
[682]	valid_0's l2: 0.00948142
[683]	valid_0's l2: 0.0094571
[684]	valid_0's l2: 0.00942921
[685]	valid_0's l2: 0.00940303
[686]	valid_0's l2: 0.00937365
[687]	valid_0's l2: 0.00934648
[688]	valid_0's l2: 0.00931874
[689]	valid_0's l2: 0.009291
[690]	valid_0's l2: 0.00926219
[691]	valid_0's l2: 0.00923635
[692]	valid_0's l2: 0.00921003
[693]	valid_0's l2: 0.00918178
[694]	valid_0's l2: 0.00915312
[695]	valid_0's l2: 0.00912756
[696]	valid_0's l2: 0.00910119
[697]	valid_0's l2: 0.00907177
[698]	valid_0's l2: 0.00904345
[699]	valid_0's l2: 0.00901829
[700]	valid_0's l2: 0.00899257
[701]	valid_0's l2: 0.0089655
[702]	valid_0's l2: 0.0089406
[703]	valid_0's l2: 0.00891661
[704]	valid_0's l2: 0.00889277
[705]	valid_0's l2: 0.00886283
[706]	valid_0's l2: 0.00883454
[707]	valid_0's l2: 0.0088102
[708]	valid_0's l2: 0.00878652
[709]	valid_0's l2: 0.00875958
[710]	valid_0's l2: 0.00873833
[711]	valid_0's l2: 0.00871026
[712]	valid_0's l2: 0.00868098
[713]	valid_0's l2: 0.00865403
[714]	valid_0's l2: 0.00862837
[715]	valid_0's l2: 0.00860197
[716]	valid_0's l2: 0.00857628
[717]	valid_0's l2: 0.0085533
[718]	valid_0's l2: 0.00852698
[719]	valid_0's l2: 0.00850126
[720]	valid_0's l2: 0.00847684
[721]	valid_0's l2: 0.00844918
[722]	valid_0's l2: 0.00842668
[723]	valid_0's l2: 0.00840213
[724]	valid_0's l2: 0.00837805
[725]	valid_0's l2: 0.00834966
[726]	valid_0's l2: 0.00832772
[727]	valid_0's l2: 0.0083058
[728]	valid_0's l2: 0.00828248
[729]	valid_0's l2: 0.00825812
[730]	valid_0's l2: 0.00823344
[731]	valid_0's l2: 0.00820943
[732]	valid_0's l2: 0.0081886
[733]	valid_0's l2: 0.00816706
[734]	valid_0's l2: 0.00813851
[735]	valid_0's l2: 0.00811264
[736]	valid_0's l2: 0.00808684
[737]	valid_0's l2: 0.00805964
[738]	valid_0's l2: 0.00803515
[739]	valid_0's l2: 0.0080105
[740]	valid_0's l2: 0.00798574
[741]	valid_0's l2: 0.00795922
[742]	valid_0's l2: 0.00793924
[743]	valid_0's l2: 0.00792129
[744]	valid_0's l2: 0.00789851
[745]	valid_0's l2: 0.00787613
[746]	valid_0's l2: 0.00785161
[747]	valid_0's l2: 0.00782597
[748]	valid_0's l2: 0.00780589
[749]	valid_0's l2: 0.00778356
[750]	valid_0's l2: 0.00776195
[751]	valid_0's l2: 0.0077431
[752]	valid_0's l2: 0.00772208
[753]	valid_0's l2: 0.00770088
[754]	valid_0's l2: 0.00767512
[755]	valid_0's l2: 0.00765321
[756]	valid_0's l2: 0.00762882
[757]	valid_0's l2: 0.00760725
[758]	valid_0's l2: 0.00758567
[759]	valid_0's l2: 0.00756528
[760]	valid_0's l2: 0.00754057
[761]	valid_0's l2: 0.00751887
[762]	valid_0's l2: 0.00749747
[763]	valid_0's l2: 0.00747596
[764]	valid_0's l2: 0.00745263
[765]	valid_0's l2: 0.0074307
[766]	valid_0's l2: 0.00741019
[767]	valid_0's l2: 0.00739204
[768]	valid_0's l2: 0.00737176
[769]	valid_0's l2: 0.00734942
[770]	valid_0's l2: 0.00732936
[771]	valid_0's l2: 0.00730547
[772]	valid_0's l2: 0.00728343
[773]	valid_0's l2: 0.00726527
[774]	valid_0's l2: 0.00725046
[775]	valid_0's l2: 0.007226
[776]	valid_0's l2: 0.00720724
[777]	valid_0's l2: 0.00719013
[778]	valid_0's l2: 0.00716942
[779]	valid_0's l2: 0.00714917
[780]	valid_0's l2: 0.00712796
[781]	valid_0's l2: 0.00710613
[782]	valid_0's l2: 0.0070854
[783]	valid_0's l2: 0.007067
[784]	valid_0's l2: 0.00704738
[785]	valid_0's l2: 0.00702729
[786]	valid_0's l2: 0.00700954
[787]	valid_0's l2: 0.00698751
[788]	valid_0's l2: 0.00696706
[789]	valid_0's l2: 0.00695013
[790]	valid_0's l2: 0.00692799
[791]	valid_0's l2: 0.00690834
[792]	valid_0's l2: 0.00689022
[793]	valid_0's l2: 0.00687461
[794]	valid_0's l2: 0.00685537
[795]	valid_0's l2: 0.00683864
[796]	valid_0's l2: 0.00681821
[797]	valid_0's l2: 0.00679957
[798]	valid_0's l2: 0.00678304
[799]	valid_0's l2: 0.00676526
[800]	valid_0's l2: 0.00674829
[801]	valid_0's l2: 0.00672748
[802]	valid_0's l2: 0.00671314
[803]	valid_0's l2: 0.00669539
[804]	valid_0's l2: 0.00667696
[805]	valid_0's l2: 0.00666189
[806]	valid_0's l2: 0.00664278
[807]	valid_0's l2: 0.00662476
[808]	valid_0's l2: 0.0066045
[809]	valid_0's l2: 0.00658688
[810]	valid_0's l2: 0.00657008
[811]	valid_0's l2: 0.00654931
[812]	valid_0's l2: 0.00652892
[813]	valid_0's l2: 0.00650945
[814]	valid_0's l2: 0.00649235
[815]	valid_0's l2: 0.00647223
[816]	valid_0's l2: 0.00645115
[817]	valid_0's l2: 0.00643335
[818]	valid_0's l2: 0.00641364
[819]	valid_0's l2: 0.00639329
[820]	valid_0's l2: 0.00637625
[821]	valid_0's l2: 0.00635915
[822]	valid_0's l2: 0.00633663
[823]	valid_0's l2: 0.00631993
[824]	valid_0's l2: 0.0063028
[825]	valid_0's l2: 0.0062848
[826]	valid_0's l2: 0.00626578
[827]	valid_0's l2: 0.00624708
[828]	valid_0's l2: 0.00623038
[829]	valid_0's l2: 0.00621295
[830]	valid_0's l2: 0.00619585
[831]	valid_0's l2: 0.00618205
[832]	valid_0's l2: 0.00616672
[833]	valid_0's l2: 0.00614851
[834]	valid_0's l2: 0.00613162
[835]	valid_0's l2: 0.00611457
[836]	valid_0's l2: 0.00609737
[837]	valid_0's l2: 0.00608204
[838]	valid_0's l2: 0.00606595
[839]	valid_0's l2: 0.00604497
[840]	valid_0's l2: 0.00602921
[841]	valid_0's l2: 0.00601185
[842]	valid_0's l2: 0.00599524
[843]	valid_0's l2: 0.00597993
[844]	valid_0's l2: 0.00596425
[845]	valid_0's l2: 0.00594689
[846]	valid_0's l2: 0.00592871
[847]	valid_0's l2: 0.0059126
[848]	valid_0's l2: 0.00589392
[849]	valid_0's l2: 0.00587897
[850]	valid_0's l2: 0.00586262
[851]	valid_0's l2: 0.00584732
[852]	valid_0's l2: 0.00582891
[853]	valid_0's l2: 0.00581132
[854]	valid_0's l2: 0.00579666
[855]	valid_0's l2: 0.00577848
[856]	valid_0's l2: 0.00576402
[857]	valid_0's l2: 0.00574758
[858]	valid_0's l2: 0.00572844
[859]	valid_0's l2: 0.00571458
[860]	valid_0's l2: 0.00570056
[861]	valid_0's l2: 0.00568792
[862]	valid_0's l2: 0.00567322
[863]	valid_0's l2: 0.00565784
[864]	valid_0's l2: 0.00564145
[865]	valid_0's l2: 0.00562681
[866]	valid_0's l2: 0.00560813
[867]	valid_0's l2: 0.00559159
[868]	valid_0's l2: 0.00557886
[869]	valid_0's l2: 0.00556378
[870]	valid_0's l2: 0.00554823
[871]	valid_0's l2: 0.00553179
[872]	valid_0's l2: 0.00551684
[873]	valid_0's l2: 0.00550021
[874]	valid_0's l2: 0.0054866
[875]	valid_0's l2: 0.00547274
[876]	valid_0's l2: 0.00545589
[877]	valid_0's l2: 0.00544012
[878]	valid_0's l2: 0.0054281
[879]	valid_0's l2: 0.00541436
[880]	valid_0's l2: 0.00540007
[881]	valid_0's l2: 0.00538562
[882]	valid_0's l2: 0.00537077
[883]	valid_0's l2: 0.00535535
[884]	valid_0's l2: 0.00534103
[885]	valid_0's l2: 0.00532573
[886]	valid_0's l2: 0.00530942
[887]	valid_0's l2: 0.00529119
[888]	valid_0's l2: 0.00527625
[889]	valid_0's l2: 0.00526415
[890]	valid_0's l2: 0.00525031
[891]	valid_0's l2: 0.00523463
[892]	valid_0's l2: 0.00522234
[893]	valid_0's l2: 0.00520553
[894]	valid_0's l2: 0.00519216
[895]	valid_0's l2: 0.00517861
[896]	valid_0's l2: 0.0051687
[897]	valid_0's l2: 0.00515267
[898]	valid_0's l2: 0.00514124
[899]	valid_0's l2: 0.00512636
[900]	valid_0's l2: 0.00511042
[901]	valid_0's l2: 0.00509234
[902]	valid_0's l2: 0.00507939
[903]	valid_0's l2: 0.00506686
[904]	valid_0's l2: 0.00505396
[905]	valid_0's l2: 0.00503911
[906]	valid_0's l2: 0.00502585
[907]	valid_0's l2: 0.00501396
[908]	valid_0's l2: 0.00500107
[909]	valid_0's l2: 0.00498863
[910]	valid_0's l2: 0.00497211
[911]	valid_0's l2: 0.00496041
[912]	valid_0's l2: 0.00494742
[913]	valid_0's l2: 0.00493091
[914]	valid_0's l2: 0.00491638
[915]	valid_0's l2: 0.00490328
[916]	valid_0's l2: 0.00488862
[917]	valid_0's l2: 0.00487562
[918]	valid_0's l2: 0.00486164
[919]	valid_0's l2: 0.00484643
[920]	valid_0's l2: 0.00483327
[921]	valid_0's l2: 0.00482031
[922]	valid_0's l2: 0.00481015
[923]	valid_0's l2: 0.00479466
[924]	valid_0's l2: 0.00478117
[925]	valid_0's l2: 0.00476855
[926]	valid_0's l2: 0.00475397
[927]	valid_0's l2: 0.00473896
[928]	valid_0's l2: 0.00472724
[929]	valid_0's l2: 0.00471297
[930]	valid_0's l2: 0.00470006
[931]	valid_0's l2: 0.00468702
[932]	valid_0's l2: 0.00467521
[933]	valid_0's l2: 0.00466514
[934]	valid_0's l2: 0.00465004
[935]	valid_0's l2: 0.0046374
[936]	valid_0's l2: 0.00462535
[937]	valid_0's l2: 0.00461603
[938]	valid_0's l2: 0.00460596
[939]	valid_0's l2: 0.00459329
[940]	valid_0's l2: 0.00458267
[941]	valid_0's l2: 0.00457171
[942]	valid_0's l2: 0.00456195
[943]	valid_0's l2: 0.00454889
[944]	valid_0's l2: 0.00453498
[945]	valid_0's l2: 0.0045241
[946]	valid_0's l2: 0.00451257
[947]	valid_0's l2: 0.00450106
[948]	valid_0's l2: 0.00449129
[949]	valid_0's l2: 0.00447776
[950]	valid_0's l2: 0.00446848
[951]	valid_0's l2: 0.00445618
[952]	valid_0's l2: 0.00444714
[953]	valid_0's l2: 0.00443571
[954]	valid_0's l2: 0.00442496
[955]	valid_0's l2: 0.0044127
[956]	valid_0's l2: 0.00440143
[957]	valid_0's l2: 0.00438998
[958]	valid_0's l2: 0.0043749
[959]	valid_0's l2: 0.00436525
[960]	valid_0's l2: 0.00435274
[961]	valid_0's l2: 0.00434034
[962]	valid_0's l2: 0.00432844
[963]	valid_0's l2: 0.0043179
[964]	valid_0's l2: 0.00430632
[965]	valid_0's l2: 0.00429557
[966]	valid_0's l2: 0.0042838
[967]	valid_0's l2: 0.00427301
[968]	valid_0's l2: 0.00426204
[969]	valid_0's l2: 0.00425168
[970]	valid_0's l2: 0.0042391
[971]	valid_0's l2: 0.00422889
[972]	valid_0's l2: 0.00421565
[973]	valid_0's l2: 0.00420352
[974]	valid_0's l2: 0.00419274
[975]	valid_0's l2: 0.00418182
[976]	valid_0's l2: 0.00417131
[977]	valid_0's l2: 0.00415949
[978]	valid_0's l2: 0.00414624
[979]	valid_0's l2: 0.00413553
[980]	valid_0's l2: 0.00412225
[981]	valid_0's l2: 0.00411082
[982]	valid_0's l2: 0.00409779
[983]	valid_0's l2: 0.00408897
[984]	valid_0's l2: 0.00407828
[985]	valid_0's l2: 0.00406799
[986]	valid_0's l2: 0.00405855
[987]	valid_0's l2: 0.00404701
[988]	valid_0's l2: 0.00403567
[989]	valid_0's l2: 0.00402557
[990]	valid_0's l2: 0.00401755
[991]	valid_0's l2: 0.00400607
[992]	valid_0's l2: 0.00399713
[993]	valid_0's l2: 0.00398729
[994]	valid_0's l2: 0.00397694
[995]	valid_0's l2: 0.00396689
[996]	valid_0's l2: 0.00395627
[997]	valid_0's l2: 0.0039451
[998]	valid_0's l2: 0.00393505
[999]	valid_0's l2: 0.00392469
[1000]	valid_0's l2: 0.00391117
Did not meet early stopping. Best iteration is:
[1000]	valid_0's l2: 0.00391117
pred
[[ 0.47436676]
 [ 0.37895368]
 [ 0.08911858]
 ...
 [-2.55487643]
 [-2.56092274]
 [-2.54996901]]
[[ 0.47436676]
 [ 0.37895368]
 [ 0.08911858]
 ...
 [-2.55487643]
 [-2.56092274]
 [-2.54996901]]

ZShiJ

关注

43
点赞
踩
20

收藏

觉得还不错? 一键收藏
打赏
0
评论
机器学习实战 —— 工业蒸汽量预测（六）

燃料在燃烧时加热水生成蒸汽，蒸汽压力推动汽轮机旋转，然后汽轮机带动发电机旋转，产生电能。在这一系列的能量转化中，影响发电效率的核心是锅炉的燃烧效率，即燃料燃烧加热水产生高温高压蒸汽。数据分析：查看变量间相关性以及找出关键变量。数据特征工程对数据精进：异常值处理、归一化处理以及特征降维。模型训练(涉及主流ML模型)：决策树、随机森林，lightgbm等。模型验证：评估指标以及交叉验证等。特征优化：用lgb对特征进行优化。模型融合：进行基于stacking方式模型融合。
复制链接

扫一扫