一起挖掘幸福感赛事学习

最新推荐文章于 2021-09-16 18:06:43 发布
最新推荐文章于 2021-09-16 18:06:43 发布
阅读量209 收藏
点赞数
分类专栏： 机器学习
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/hutugui2009/article/details/111633123
版权

幸福感预测线上0.47593

import sys
import gc
import pandas as pd
import matplotlib
import numpy as np
import scipy as sp
from scipy import stats
from scipy.stats import norm,skew,kurtosis #for some statistics
import IPython
from IPython import display
import sklearn
import random
import time
import pickle
from datetime import datetime
import matplotlib.pyplot as plt
import seaborn as sns
from pylab import mpl
mpl.rcParams['font.sans-serif'] = ['FangSong'] # 指定默认字体
mpl.rcParams['axes.unicode_minus'] = False # 解决保存图像是负号'-'显示为方块的问题
%matplotlib inline
#ignore warnings
import warnings
warnings.filterwarnings('ignore')
print('-'*25)
help(pd.read_csv)

input_path = 'E:\Document\aliyun_tianchi\task 04'
train_data = pd.read_csv(r'E:\Document\aliyun_tianchi\task 04\happiness_train_complete.csv',sep=',',encoding='latin-1')
test_data = pd.read_csv(r'E:\Document\aliyun_tianchi\task 04\happiness_test_complete.csv',sep=',',encoding='latin-1')
submit_example = pd.read_csv(r'E:\Document\aliyun_tianchi\task 04\happiness_submit.csv',sep=',',encoding='latin-1')


print('train shape:',train_data.shape)
print('test shape:',test_data.shape)
print('sample shape:',submit_example.shape)
#print(train_data)

train shape: (8000, 140)
test shape: (2968, 139)
sample shape: (2968, 2)


train_data = train_data[train_data["happiness"]!=-8].reset_index(drop=True)#注意drop=True  
print('train shape:',train_data.shape)
train_data_copy = train_data.copy()
target_col = "happiness"
target = train_data_copy[target_col]#.apply(lambda x:np.log1p(x))

del train_data_copy[target_col]

train_shape = train_data.shape[0]

data = pd.concat([train_data_copy,test_data],axis=0,ignore_index=True) #合并测试集与训练集
data.head()
print('train shape:',data.shape)


train shape: (7988, 140)
train shape: (10956, 139)

#make feature
def getres1(row):
    return len([x for x in row.values if type(x)==int and x<0])

def getres2(row):
    return len([x for x in row.values if type(x)==int and x==-8])

def getres3(row):
    return len([x for x in row.values if type(x)==int and x==-1])

def getres4(row):
    return len([x for x in row.values if type(x)==int and x==-2])

def getres5(row):
    return len([x for x in row.values if type(x)==int and x==-3])

#调查问卷有效程度
data['neg1'] = data[data.columns].apply(lambda row:getres1(row),axis=1)
data.loc[data['neg1']>20,'neg1'] = 20  #平滑处理

print(data)

391      27
751      31
1110     25
1776     25
2049     27
2059     23
3872     23
4263     26
4641     24
4654     24
4722     48
4987     23
5284     22
5604     27
5806     33
5861     21
5943     21
6047     28
6094     23
6253     29
6381     24
6589     22
6927     21
7440     24
7522     21
8550     21
8597     22
8789     21
9028     21
9029     23
9061     30
9148     22
9355     21
9687     22
10054    21
10064    21
10112    24
10121    21
10387    27
10596    28
10649    21
Name: neg1, dtype: int64

#make feature
def getres1(row):
    return len([x for x in row.values if type(x)==int and x<0])

def getres2(row):
    return len([x for x in row.values if type(x)==int and x==-8])

def getres3(row):
    return len([x for x in row.values if type(x)==int and x==-1])

def getres4(row):
    return len([x for x in row.values if type(x)==int and x==-2])

def getres5(row):
    return len([x for x in row.values if type(x)==int and x==-3])

#调查问卷有效程度
data['neg1'] = data[data.columns].apply(lambda row:getres1(row),axis=1)
data.loc[data['neg1']>20,'neg1'] = 20  #平滑处理

data['neg2'] = data[data.columns].apply(lambda row:getres2(row),axis=1)
data['neg3'] = data[data.columns].apply(lambda row:getres3(row),axis=1)
data['neg4'] = data[data.columns].apply(lambda row:getres4(row),axis=1)
data['neg5'] = data[data.columns].apply(lambda row:getres5(row),axis=1)

data.loc[data['health_problem']<0,'health_problem'] = 0
data.loc[data['religion']<0,'religion'] = 1


data.loc[data['religion_freq']<0,'religion_freq'] = 1

data.loc[data['edu']<0,'edu'] = 0

data.loc[data['edu_status']<0,'edu_status'] = 0

data.loc[data['income']<0,'income'] = 0

data.loc[data['s_income']<0,'s_income']= 0

#对体重修正

data.loc[(data['weight_jin']<=80)&(data['height_cm']>=160),'weight_jin']= data['weight_jin']*2
data.loc[data['weight_jin']<=60,'weight_jin']= data['weight_jin']*2


data.loc[data['family_income']<=0,'family_income']=0
data.loc[data['inc_exp']<=0,'inc_exp']= 0

data.loc[data['equity']<0,'equity'] = 0
data.loc[data['social_neighbor']<0,'social_neighbor'] = 0
data.loc[data['class_10_after']<0,'class_10_after'] = 0
data.loc[data['class_10_before']<0,'class_10_before'] = 0
data.loc[data['class']<0,'class'] = 0
data.loc[data['class_14']<0,'class_14'] = 0
data.loc[data['family_m']<0,'family_m'] = 1
data.loc[data['health']<0,'health'] = 0
data.loc[data['health_problem']<0,'health_problem'] = 0
data.loc[data['neighbor_familiarity']<0,'neighbor_familiarity'] = 0
data.loc[data['inc_ability']<0,'inc_ability'] = np.nan
data.loc[data['status_peer']<0,'status_peer'] = 2  ##
data.loc[data['status_3_before']<0,'status_3_before'] = 2  ##
data.loc[data['edu_yr']<0,'edu_yr'] = 0

#部分特征处理
#data.loc[data['marital_1st']<0,'marital_1st']= np.nan
#data['survey_time'] = pd.to_datetime(data['survey_time'])
#data.loc[data['marital_now']<0,'marital_now']= np.nan
#data.loc[data['join_party']<0,'join_party']= np.nan
#for i in range(1,12+1):
#    data.loc[data['leisure_'+str(i)]<0,'leisure_'+str(i)] = 6
#for i in range(1,9+1):
#    data.loc[data['public_service_'+str(i)]<0,'public_service_'+str(i)] = data['public_service_'+str(i)].dropna().mode().values
#for i in range(1,13+1):
#    data.loc[data['trust_'+str(i)]<0,'trust_'+str(i)] = data['trust_'+str(i)].dropna().mode().values    


#毕业年龄
data['edubir'] = data['edu_yr'] - data['birth'] 
#调查年龄
data['survey_age'] = 2015-data['birth']
#毕业后调查年数差
data['survey_edu_age'] = 2015-data['edu_yr']

#调查月、时
data['survey_month'] = data['survey_time'].dt.month
data['survey_hour']= data['survey_time'].dt.hour
#第一次结婚年龄
data['marital_1stbir'] = data['marital_1st'] - data['birth'] 
#最近结婚年龄
data['marital_nowtbir'] = data['marital_now'] - data['birth'] 
#是否再婚
data['mar'] = data['marital_nowtbir'] - data['marital_1stbir']
#配偶年龄
data['marital_sbir'] = data['marital_now']-data['s_birth']
#配偶年龄差
data['age_'] = data['marital_nowtbir'] - data['marital_sbir'] 

#收入比
data['income/s_income'] = data['income']/(data['s_income']+1)
data['income+s_income'] = data['income']+(data['s_income']+1)
data['income/family_income'] = data['income']/(data['family_income']+1)
data['all_income/family_income'] = (data['income']+data['s_income'])/(data['family_income']+1)
data['income/inc_exp'] = data['income']/(data['inc_exp']+1)
data['family_income/m'] = data['family_income']/(data['family_m']+0.01)

#收入/面积比
data['income/floor_area'] = data['income']/(data['floor_area']+0.01)
data['all_income/floor_area'] = (data['income']+data['s_income'])/(data['floor_area']+0.01)
data['family_income/floor_area'] = data['family_income']/(data['floor_area']+0.01)

data['income/m'] = data['floor_area']/(data['family_m']+0.01)

#class
data['class_10_diff'] = (data['class_10_after'] - data['class_10_before'])
data['class_diff'] = data['class'] - data['class_10_before']
data['class_14_diff'] = data['class'] - data['class_14']


#province mean
data['province_income_mean'] = data.groupby(['province'])['income'].transform('mean').values
data['province_family_income_mean'] = data.groupby(['province'])['family_income'].transform('mean').values
data['province_equity_mean'] = data.groupby(['province'])['equity'].transform('mean').values
data['province_depression_mean'] = data.groupby(['province'])['depression'].transform('mean').values
data['province_floor_area_mean'] = data.groupby(['province'])['floor_area'].transform('mean').values

#city   mean
data['city_income_mean'] = data.groupby(['city'])['income'].transform('mean').values
data['city_family_income_mean'] = data.groupby(['city'])['family_income'].transform('mean').values
data['city_equity_mean'] = data.groupby(['city'])['equity'].transform('mean').values
data['city_depression_mean'] = data.groupby(['city'])['depression'].transform('mean').values
data['city_floor_area_mean'] = data.groupby(['city'])['floor_area'].transform('mean').values

#county  mean
data['county_income_mean'] = data.groupby(['county'])['income'].transform('mean').values
data['county_family_income_mean'] = data.groupby(['county'])['family_income'].transform('mean').values
data['county_equity_mean'] = data.groupby(['county'])['equity'].transform('mean').values
data['county_depression_mean'] = data.groupby(['county'])['depression'].transform('mean').values
data['county_floor_area_mean'] = data.groupby(['county'])['floor_area'].transform('mean').values

#ratio 相比同个城市
data['income/county'] = data['income']/(data['county_income_mean']+1)                                      
data['family_income/county'] = data['family_income']/(data['county_family_income_mean']+1)   
data['equity/county'] = data['equity']/(data['county_equity_mean']+1)       
data['depression/county'] = data['depression']/(data['county_depression_mean']+1)                                                
data['floor_area/county'] = data['floor_area']/(data['county_floor_area_mean']+1)   

#age   mean
data['age_income_mean'] = data.groupby(['survey_age'])['income'].transform('mean').values
data['age_family_income_mean'] = data.groupby(['survey_age'])['family_income'].transform('mean').values
data['age_equity_mean'] = data.groupby(['survey_age'])['equity'].transform('mean').values
data['age_depression_mean'] = data.groupby(['survey_age'])['depression'].transform('mean').values
data['age_floor_area_mean'] = data.groupby(['survey_age'])['floor_area'].transform('mean').values
data['age_health_mean'] = data.groupby(['survey_age','gender'])['health'].transform('mean').values
data['age_edu_mean'] = data.groupby(['survey_age','gender'])['edu'].transform('mean').values



#age/gender   mean
data['age_income_mean'] = data.groupby(['survey_age','gender'])['income'].transform('mean').values
data['age_family_income_mean'] = data.groupby(['survey_age','gender'])['family_income'].transform('mean').values
data['age_equity_mean'] = data.groupby(['survey_age','gender'])['equity'].transform('mean').values
data['age_depression_mean'] = data.groupby(['survey_age','gender'])['depression'].transform('mean').values
data['age_floor_area_mean'] = data.groupby(['survey_age','gender'])['floor_area'].transform('mean').values
# data['age_BMI_mean'] = data.groupby(['survey_age','gender'])['BMI'].transform('mean').values
data['age_gender_health_mean'] = data.groupby(['survey_age','gender'])['health'].transform('mean').values


#class   mean
data['city_class_income_mean'] = data.groupby(['class'])['income'].transform('mean').values
data['city_class_family_income_mean'] = data.groupby(['class'])['family_income'].transform('mean').values
data['city_class_equity_mean'] = data.groupby(['class'])['equity'].transform('mean').values
data['city_class_depression_mean'] = data.groupby(['class'])['depression'].transform('mean').values
data['city_class_floor_area_mean'] = data.groupby(['class'])['floor_area'].transform('mean').values


#悠闲指数
leisure_fea_lis = ['leisure_'+str(i) for i in range(1,13)]
data['leisure_sum'] = data[leisure_fea_lis].sum() #skew

#满意指数
public_service_fea_lis = ['public_service_'+str(i) for i in range(1,10)]
data['public_service_'] = data[public_service_fea_lis].sum() #skew

data['city_public_service__mean'] = data.groupby(['city'])['public_service_'].transform('mean').values
data['public_service_cit'] = data['public_service_']-data['city_public_service__mean'] 


#信任指数
trust_fea_lis = ['trust_'+str(i) for i in range(1,10)]
data['trust_'] = data[trust_fea_lis].sum() #skew


#入党时间   
#data['survey_edu_age'] = 2015-data['join_party']
#data['survey_edu_age'].fillna(0,inplace=True)
#del data['join_party']
#del data['property_other']
#del data['id']

print(data.columns)

Index(['survey_type', 'province', 'city', 'county', 'survey_time', 'gender',
       'birth', 'nationality', 'religion', 'religion_freq',
       ...
       'city_class_income_mean', 'city_class_family_income_mean',
       'city_class_equity_mean', 'city_class_depression_mean',
       'city_class_floor_area_mean', 'leisure_sum', 'public_service_',
       'city_public_service__mean', 'public_service_cit', 'trust_'],
      dtype='object', length=202)

data.fillna(-1,inplace=True)
print('shape',data.shape)
data.head()

shape (10956, 202)
survey_typeprovincecitycountysurvey_timegenderbirthnationalityreligionreligion_freq...city_class_income_meancity_class_family_income_meancity_class_equity_meancity_class_depression_meancity_class_floor_area_meanleisure_sumpublic_service_city_public_service__meanpublic_service_cittrust_
011232592015-08-04 14:18:0011959111...24641.04081639486.6435373.0265313.650340107.158027-1.0-1.0-1.0-1.0-1.0
121852852015-07-21 15:04:0011992111...43440.000000104062.5761643.3378064.023678124.915075-1.0-1.0-1.0-1.0-1.0
2229831262015-07-21 13:24:0021967103...32394.18320670886.4764413.2813903.943143122.765315-1.0-1.0-1.0-1.0-1.0
321028512015-07-25 17:33:0021943111...32394.18320670886.4764413.2813903.943143122.765315-1.0-1.0-1.0-1.0-1.0
41718362015-08-10 09:50:0021994111...15683.06508931480.6990522.7325443.36331497.374911-1.0-1.0-1.0-1.0-1.0

5 rows × 202 columns

#使用所有特征
use_fea = [clo for clo in data.columns if clo!='survey_time' and data[clo].dtype!=object]
features = data[use_fea].columns
X_train = data[:train_shape][use_fea].values
y_train = target
X_test = data[train_shape:][use_fea].values

from sklearn.metrics import roc_auc_score, roc_curve, mean_squared_error,mean_absolute_error, f1_score
import lightgbm as lgb
import xgboost as xgb
import os
from sklearn.linear_model import BayesianRidge
from sklearn.model_selection import  KFold, StratifiedKFold,GroupKFold, RepeatedKFold
import logging

##### lgb
param = {
'num_leaves': 80,
'min_data_in_leaf': 40,
'objective':'regression',
'max_depth': -1,
'learning_rate': 0.1,
"min_child_samples": 30,
"boosting": "gbdt",
"feature_fraction": 0.9,
"bagging_freq": 2,
"bagging_fraction": 0.9,
"bagging_seed": 2029,
"metric": 'mse',
"lambda_l1": 0.1,
"lambda_l2": 0.2, 
"verbosity": -1}
folds = KFold(n_splits=10, shuffle=True, random_state=1016)   #StratifiedKFold?   KFold
oof_lgb = np.zeros(len(X_train))
predictions_lgb = np.zeros(len(X_test))

for fold_, (trn_idx, val_idx) in enumerate(folds.split(X_train, y_train)):
    print("fold n°{}".format(fold_+1))
    trn_data = lgb.Dataset(X_train[trn_idx], y_train[trn_idx])
    val_data = lgb.Dataset(X_train[val_idx], y_train[val_idx])

    num_round = 10000
    clf = lgb.train(param, trn_data, num_round, valid_sets = [trn_data, val_data], verbose_eval=200, early_stopping_rounds = 200)
    oof_lgb[val_idx] = clf.predict(X_train[val_idx], num_iteration=clf.best_iteration)
    predictions_lgb += clf.predict(X_test, num_iteration=clf.best_iteration) / folds.n_splits

print("CV score: {:<8.8f}".format(mean_squared_error(oof_lgb, target)))

fold n°1
[LightGBM] [Warning] min_data_in_leaf is set=40, min_child_samples=30 will be ignored. Current value: min_data_in_leaf=40
Training until validation scores don't improve for 200 rounds
[200]	training's l2: 0.0178584	valid_1's l2: 0.440415
Early stopping, best iteration is:
[53]	training's l2: 0.17148	valid_1's l2: 0.42822
fold n°2
[LightGBM] [Warning] min_data_in_leaf is set=40, min_child_samples=30 will be ignored. Current value: min_data_in_leaf=40
Training until validation scores don't improve for 200 rounds
[200]	training's l2: 0.0172029	valid_1's l2: 0.471802
Early stopping, best iteration is:
[78]	training's l2: 0.112334	valid_1's l2: 0.465844
fold n°3
[LightGBM] [Warning] min_data_in_leaf is set=40, min_child_samples=30 will be ignored. Current value: min_data_in_leaf=40
Training until validation scores don't improve for 200 rounds
[200]	training's l2: 0.0174407	valid_1's l2: 0.524768
Early stopping, best iteration is:
[48]	training's l2: 0.184526	valid_1's l2: 0.513957
fold n°4
[LightGBM] [Warning] min_data_in_leaf is set=40, min_child_samples=30 will be ignored. Current value: min_data_in_leaf=40
Training until validation scores don't improve for 200 rounds
[200]	training's l2: 0.016992	valid_1's l2: 0.509479
Early stopping, best iteration is:
[32]	training's l2: 0.247561	valid_1's l2: 0.496666
fold n°5
[LightGBM] [Warning] min_data_in_leaf is set=40, min_child_samples=30 will be ignored. Current value: min_data_in_leaf=40
Training until validation scores don't improve for 200 rounds
[200]	training's l2: 0.017454	valid_1's l2: 0.453909
Early stopping, best iteration is:
[44]	training's l2: 0.200869	valid_1's l2: 0.441661
fold n°6
[LightGBM] [Warning] min_data_in_leaf is set=40, min_child_samples=30 will be ignored. Current value: min_data_in_leaf=40
Training until validation scores don't improve for 200 rounds
[200]	training's l2: 0.0171639	valid_1's l2: 0.468323
Early stopping, best iteration is:
[60]	training's l2: 0.149475	valid_1's l2: 0.462298
fold n°7
[LightGBM] [Warning] min_data_in_leaf is set=40, min_child_samples=30 will be ignored. Current value: min_data_in_leaf=40
Training until validation scores don't improve for 200 rounds
[200]	training's l2: 0.0172571	valid_1's l2: 0.455683
Early stopping, best iteration is:
[39]	training's l2: 0.219661	valid_1's l2: 0.447699
fold n°8
[LightGBM] [Warning] min_data_in_leaf is set=40, min_child_samples=30 will be ignored. Current value: min_data_in_leaf=40
Training until validation scores don't improve for 200 rounds
[200]	training's l2: 0.0172798	valid_1's l2: 0.528483
Early stopping, best iteration is:
[32]	training's l2: 0.249316	valid_1's l2: 0.507831
fold n°9
[LightGBM] [Warning] min_data_in_leaf is set=40, min_child_samples=30 will be ignored. Current value: min_data_in_leaf=40
Training until validation scores don't improve for 200 rounds
[200]	training's l2: 0.0174847	valid_1's l2: 0.469453
Early stopping, best iteration is:
[47]	training's l2: 0.191068	valid_1's l2: 0.450083
fold n°10
[LightGBM] [Warning] min_data_in_leaf is set=40, min_child_samples=30 will be ignored. Current value: min_data_in_leaf=40
Training until validation scores don't improve for 200 rounds
[200]	training's l2: 0.0172514	valid_1's l2: 0.46314
Early stopping, best iteration is:
[74]	training's l2: 0.119155	valid_1's l2: 0.456972
CV score: 0.46712643

#---------------特征重要性
pd.set_option('display.max_columns', None)
#显示所有行
pd.set_option('display.max_rows', None)
#设置value的显示长度为100，默认为50
pd.set_option('max_colwidth',100)
df = pd.DataFrame(data[use_fea].columns.tolist(), columns=['feature'])
df['importance']=clf.feature_importance()
df = df.sort_values(by='importance',ascending=False)
plt.figure(figsize=(14,28))
sns.barplot(x="importance", y="feature", data=df.head(50))
plt.title('Features importance (averaged/folds)')
plt.tight_layout()

在这里插入图片描述

##### xgb
xgb_params = {'eta': 0.05, 'max_depth': 10, 'subsample': 0.8, 'colsample_bytree': 0.8, 
          'objective': 'reg:linear', 'eval_metric': 'rmse', 'silent': True, 'nthread': 4}

X_train_use = data[use_fea][:train_shape].values
X_test_use = data[use_fea][train_shape:].values

folds = KFold(n_splits=5, shuffle=True, random_state=2018)
oof_xgb = np.zeros(train_shape)
predictions_xgb = np.zeros(len(X_test_use))

for fold_, (trn_idx, val_idx) in enumerate(folds.split(X_train, y_train)):
    print("fold n°{}".format(fold_+1))
    trn_data = xgb.DMatrix(X_train[trn_idx], y_train[trn_idx])
    val_data = xgb.DMatrix(X_train[val_idx], y_train[val_idx])

    watchlist = [(trn_data, 'train'), (val_data, 'valid_data')]
    clf = xgb.train(dtrain=trn_data, num_boost_round=20000, evals=watchlist, early_stopping_rounds=200, verbose_eval=100, params=xgb_params)
    oof_xgb[val_idx] = clf.predict(xgb.DMatrix(X_train[val_idx]), ntree_limit=clf.best_ntree_limit)
    predictions_xgb += clf.predict(xgb.DMatrix(X_test), ntree_limit=clf.best_ntree_limit) / folds.n_splits

print("CV score: {:<8.8f}".format(mean_squared_error(oof_xgb, target)))

fold n°1
[15:02:42] WARNING: C:/Users/Administrator/workspace/xgboost-win64_release_1.3.0/src/objective/regression_obj.cu:170: reg:linear is now deprecated in favor of reg:squarederror.
[15:02:42] WARNING: C:/Users/Administrator/workspace/xgboost-win64_release_1.3.0/src/learner.cc:541: 
Parameters: { silent } might not be used.

  This may not be accurate due to some parameters are only used in language bindings but
  passed down to XGBoost core.  Or some parameters are not used but slip through this
  verification. Please open an issue if you find above cases.


[0]	train-rmse:3.30166	valid_data-rmse:3.29464
[100]	train-rmse:0.21953	valid_data-rmse:0.68739
[200]	train-rmse:0.08475	valid_data-rmse:0.68507
[300]	train-rmse:0.03203	valid_data-rmse:0.68482
[400]	train-rmse:0.01128	valid_data-rmse:0.68479
[500]	train-rmse:0.00393	valid_data-rmse:0.68485
[530]	train-rmse:0.00284	valid_data-rmse:0.68485
fold n°2
[15:03:27] WARNING: C:/Users/Administrator/workspace/xgboost-win64_release_1.3.0/src/objective/regression_obj.cu:170: reg:linear is now deprecated in favor of reg:squarederror.
[15:03:27] WARNING: C:/Users/Administrator/workspace/xgboost-win64_release_1.3.0/src/learner.cc:541: 
Parameters: { silent } might not be used.

  This may not be accurate due to some parameters are only used in language bindings but
  passed down to XGBoost core.  Or some parameters are not used but slip through this
  verification. Please open an issue if you find above cases.


[0]	train-rmse:3.29521	valid_data-rmse:3.32372
[100]	train-rmse:0.22777	valid_data-rmse:0.67676
[200]	train-rmse:0.09615	valid_data-rmse:0.67418
[300]	train-rmse:0.03693	valid_data-rmse:0.67489
[400]	train-rmse:0.01397	valid_data-rmse:0.67483
[418]	train-rmse:0.01185	valid_data-rmse:0.67480
fold n°3
[15:03:55] WARNING: C:/Users/Administrator/workspace/xgboost-win64_release_1.3.0/src/objective/regression_obj.cu:170: reg:linear is now deprecated in favor of reg:squarederror.
[15:03:55] WARNING: C:/Users/Administrator/workspace/xgboost-win64_release_1.3.0/src/learner.cc:541: 
Parameters: { silent } might not be used.

  This may not be accurate due to some parameters are only used in language bindings but
  passed down to XGBoost core.  Or some parameters are not used but slip through this
  verification. Please open an issue if you find above cases.


[0]	train-rmse:3.30791	valid_data-rmse:3.26841
[100]	train-rmse:0.22667	valid_data-rmse:0.69967
[200]	train-rmse:0.09633	valid_data-rmse:0.69768
[300]	train-rmse:0.03619	valid_data-rmse:0.69823
[376]	train-rmse:0.01673	valid_data-rmse:0.69829
fold n°4
[15:04:19] WARNING: C:/Users/Administrator/workspace/xgboost-win64_release_1.3.0/src/objective/regression_obj.cu:170: reg:linear is now deprecated in favor of reg:squarederror.
[15:04:19] WARNING: C:/Users/Administrator/workspace/xgboost-win64_release_1.3.0/src/learner.cc:541: 
Parameters: { silent } might not be used.

  This may not be accurate due to some parameters are only used in language bindings but
  passed down to XGBoost core.  Or some parameters are not used but slip through this
  verification. Please open an issue if you find above cases.


[0]	train-rmse:3.29880	valid_data-rmse:3.30782
[100]	train-rmse:0.22282	valid_data-rmse:0.69518
[200]	train-rmse:0.09137	valid_data-rmse:0.69019
[300]	train-rmse:0.03626	valid_data-rmse:0.69060
[393]	train-rmse:0.01535	valid_data-rmse:0.69049
fold n°5
[15:04:47] WARNING: C:/Users/Administrator/workspace/xgboost-win64_release_1.3.0/src/objective/regression_obj.cu:170: reg:linear is now deprecated in favor of reg:squarederror.
[15:04:47] WARNING: C:/Users/Administrator/workspace/xgboost-win64_release_1.3.0/src/learner.cc:541: 
Parameters: { silent } might not be used.

  This may not be accurate due to some parameters are only used in language bindings but
  passed down to XGBoost core.  Or some parameters are not used but slip through this
  verification. Please open an issue if you find above cases.


[0]	train-rmse:3.29689	valid_data-rmse:3.31607
[100]	train-rmse:0.22293	valid_data-rmse:0.71063
[200]	train-rmse:0.08727	valid_data-rmse:0.71013
[300]	train-rmse:0.03349	valid_data-rmse:0.71038
[338]	train-rmse:0.02331	valid_data-rmse:0.71032
CV score: 0.47779700

# 将lgb和xgb的结果进行stacking
train_stack = np.vstack([oof_lgb,oof_xgb]).transpose()
test_stack = np.vstack([predictions_lgb, predictions_xgb]).transpose()

folds_stack = RepeatedKFold(n_splits=5, n_repeats=2, random_state=4590)
oof_stack = np.zeros(train_stack.shape[0])
predictions = np.zeros(test_stack.shape[0])

for fold_, (trn_idx, val_idx) in enumerate(folds_stack.split(train_stack,target)):
    print("fold {}".format(fold_))
    trn_data, trn_y = train_stack[trn_idx], target.iloc[trn_idx].values
    val_data, val_y = train_stack[val_idx], target.iloc[val_idx].values
    
    clf_3 = BayesianRidge()
    clf_3.fit(trn_data, trn_y)
    
    oof_stack[val_idx] = clf_3.predict(val_data)
    predictions += clf_3.predict(test_stack) / 10
    
mean_squared_error(target.values, oof_stack) 

fold 0
fold 1
fold 2
fold 3
fold 4
fold 5
fold 6
fold 7
fold 8
fold 9





0.4606959700010243

submit_example['happiness'] = predictions
now = datetime.now().strftime('%m-%d-%H-%M')
submit_example.to_csv("./baseline_%s.csv" % now, index=False)
submit_example.head()
idhappiness
080013.727733
180022.828579
280033.211417
380044.362161
480053.253053
hutugui2009
关注
  • 0
    点赞
  • 踩
  • 0
    收藏
    觉得还不错? 一键收藏
  • 1
    评论
专栏目录
python语言中param_Python中XGBoost库Param使用问题。-python
weixin_39986060的博客
11-29 2739
在学习使用xgboost的过程中，编写param传入xgboost.cv等函数过程中jupyter一直提示Parameters: { xxx } might not be used.想请问出现的原因以及解决办法，代码应该是没问题的。from xgboost import XGBRegressor as XGBRfrom sklearn.ensemble import RandomForestReg...
关于xgboost的一些整理
不再的blog
08-10 8479
XGBOOST的三大板块 XGBoost本身的核心是基于梯度提升树实现的集成算法，整体来说可以有三个核心部分：集成算法本身，用于集成的 弱评估器，以及应用中的其他过程。三个部分中，前两个部分包含了XGBoost的核心原理以及数学过程，最后的部分 主要是在XGBoost应用中占有一席之地。我们的课程会主要集中在前两部分，最后一部分内容将会在应用中少量给大 家提及。接下来，我们就针对这三个部分，来进行一一的讲解 from xgboost import XGBRegressor as XGBR from skl
Parameters: { “silent“ } might not be used. This may not be accurate due to some parameters are
wangxiancao的博客
09-16 6925
项目场景： 在使用XGBoost训练时，遇到如下报错: Parameters: { "silent" } might not be used. This may not be accurate due to some parameters are only used in language bindings but passed down to XGBoost core. Or some parameters are not used but slip through this veri
基于深度学习的文本分类1-FastText
不再的blog
07-28 1949
基于深度学习的文本分类1-FastText 续基于机器学习的文本分类（上次使用了CountVector+RidgeClassifer和Tfidf+RidgeClassfier） # tfidf+xgb import numpy as np import pandas as pd from sklearn.model_selection import train_test_split from sklearn.feature_extraction.text import TfidfVectorizer fro
快来一起挖掘幸福感吧！.zip
07-25
标题“快来一起挖掘幸福感吧！”指的是一个数据挖掘项目，旨在通过分析特定的数据集来寻找与幸福感相关的模式。这个项目在阿里云天池平台上进行，这是一个知名的云端大数据竞赛平台，为数据科学家和机器学习爱好者...
【新人赛】快来一起挖掘幸福感！
07-08
在IT行业中，"【新人赛】快来一起挖掘幸福感!" 可能是指一场针对新手或入门者的编程竞赛，旨在通过解决实际问题来提升参赛者的技术能力和幸福感。这种比赛通常由技术平台如阿里云天池（Tianchi.aliyun.com）主办，...
幸福感分析-机器学习项目 https://tianchi.aliyun.com/competition/entrance/231
01-18
幸福感分析-机器学习项目 https://tianchi.aliyun.com/competition/entrance/231702/introduction 幸福感是一个古老而深刻的话题，是人类世代追求的方向。与幸福感相关的因素成千上万、因人而异，大如国计民生，...
python数据挖掘实战六个完整学习项目（代码+分析+结果+数据集.zip
最新发布
04-13
在本资源中，你将通过六个完整的Python数据挖掘学习项目深入了解数据挖掘的实践应用。这些项目涵盖了一系列领域，包括员工流失预警、电信客户流失、药物数据挖掘、世界幸福报告分析、英雄联盟比赛胜负预测以及保险业...
小学校长跟岗学习总结2020.doc
10-30
6. **教师的角色与课堂管理**：教师的成功感和幸福感源于课堂，学校鼓励教师减少讲解时间，增加学生交流和自主学习的机会，通过不同层次的练习促进学生自立。 7. **学校管理机制**：有效的管理机制可以激励教师积极...
REST API设计指导——译自Microsoft REST API Guidelines（四）
weixin_34327761的博客
11-14 571
前言 前面我们说了，如果API的设计更规范更合理，在很大程度上能够提高联调的效率，降低沟通成本。那么什么是好的API设计？这里我们不得不提到REST API。   关于REST API的书籍很多，但是完整完善实践丰富的设计指导并不多见，我们有幸看到了微软团队的作品——Microsoft REST API Guidelines，因此才有了此篇内容。   由于公众号文章内容字数有限，因此我们...
python,flask
weixin_30916125的博客
01-22 1363
API¶ This part of the documentation covers all the interfaces of Flask. For parts where Flask depends on external libraries, we document the most important right here and provide links to ...
NSIS用法注释
热门推荐
Sam.yu的专栏
03-08 1万+
<br />;this is a test comment<br /> #this is a test comment too,<br /> /*this is  a test comment too*/<br /> ;MessageBox MB_OK "hi test!"<br /> ;MessageBox MB_OK "$/"A quote from a wise man$/" said the wise man"<br /><br /> ;Var MYVAR<br /> ;StrCpy $
submit提交
枫的博客
09-25 700
<!DOCTYPE html> <html> <head lang="en"> <meta charset="UTF-8"> <title></title> <script src="lib/jquery.js"></script> <script> function abc() { $("#xx").submit(functi
数据挖掘、机器学习与深度学习解析
"数据挖掘与机器学习.pdf" 在深入探讨数据挖掘与机器学习之前，我们首先需要理解它们之间的关联和区别。数据挖掘是一个广泛的概念，它涵盖了从海量数据中提取有价值信息的各种方法。BI（商业智能）、统计分析、...
评论 1
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值