【教学赛】金融数据分析赛题1：银行客户认购产品预测（0.9676）

 本文是对天池教学赛，银行客户认购产品预测的记录，教学赛网址如下：

【教学赛】金融数据分析赛题1：银行客户认购产品预测_学习赛_天池大赛-阿里云天池

 1. 读取数据

import pandas as pd

# 加载数据
train = pd.read_csv('train.csv')
test = pd.read_csv('test.csv')

2. 数据处理

2.1 合并数据

# 训练集和测试集合并, 以便于处理特征的数据
df = pd.concat([train, test], axis=0) #将训练数据和测试数据在行的方向拼接
df

得到的结果：

id	age	job	marital	education	default	housing	loan	contact	month	...	campaign	pdays	previous	poutcome	emp_var_rate	cons_price_index	cons_conf_index	lending_rate3m	nr_employed	subscribe
0	1	51	admin.	divorced	professional.course	no	yes	yes	cellular	aug	...	1	112	2	failure	1.4	90.81	-35.53	0.69	5219.74	no
1	2	50	services	married	high.school	unknown	yes	no	cellular	may	...	1	412	2	nonexistent	-1.8	96.33	-40.58	4.05	4974.79	yes
2	3	48	blue-collar	divorced	basic.9y	no	no	no	cellular	apr	...	0	1027	1	failure	-1.8	96.33	-44.74	1.50	5022.61	no
3	4	26	entrepreneur	single	high.school	yes	yes	yes	cellular	aug	...	26	998	0	nonexistent	1.4	97.08	-35.55	5.11	5222.87	yes
4	5	45	admin.	single	university.degree	no	no	no	cellular	nov	...	1	240	4	success	-3.4	89.82	-33.83	1.17	4884.70	no
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
7495	29996	49	admin.	unknown	university.degree	unknown	yes	yes	telephone	apr	...	50	302	1	failure	-1.8	95.77	-40.50	3.86	5058.64	NaN
7496	29997	34	blue-collar	married	basic.4y	no	no	no	cellular	jul	...	8	440	3	failure	1.4	90.59	-47.29	1.77	5156.70	NaN
7497	29998	50	retired	single	basic.4y	no	yes	no	cellular	jun	...	3	997	0	nonexistent	-2.9	97.42	-39.69	1.29	5116.80	NaN
7498	29999	31	technician	married	professional.course	no	no	no	cellular	aug	...	3	1028	0	nonexistent	1.4	96.90	-37.68	5.18	5144.45	NaN
7499	30000	46	admin.	divorced	university.degree	no	yes	no	cellular	aug	...	2	387	3	success	1.4	97.49	-31.54	3.79	5082.25	NaN
30000 rows × 22 columns

可见数据既有数字也有文字，需要将文字转换为数字

2.2 将非数字的特征转换为数字

# 首先选出所有的特征为object(非数字)的特征
cat_columns = df.select_dtypes(include='object').columns  #选择非数字的列，对其进行处理
df[cat_columns]

# 对非数字特征进行编码
from sklearn.preprocessing import LabelEncoder

job_le = LabelEncoder()
df['job'] = job_le.fit_transform(df['job'])
df['marital'] = df['marital'].map({'unknown':0, 'single':1, 'married':2, 'divorced':3})
df['education'] = df['education'].map({'unknown':0, 'basic.4y':1, 'basic.6y':2, 'basic.9y':3, 'high.school':4, 'university.degree':5, 'professional.course':6, 'illiterate':7})
df['housing'] = df['housing'].map({'unknown': 0, 'no': 1, 'yes': 2})
df['loan'] = df['loan'].map({'unknown': 0, 'no': 1, 'yes': 2})
df['contact'] = df['contact'].map({'cellular': 0, 'telephone': 1})
df['day_of_week'] = df['day_of_week'].map({'mon': 0, 'tue': 1, 'wed': 2, 'thu': 3, 'fri': 4})
df['poutcome'] = df['poutcome'].map({'nonexistent': 0, 'failure': 1, 'success': 2})
df['default'] = df['default'].map({'unknown': 0, 'no': 1, 'yes': 2})
df['month'] = df['month'].map({'mar': 3, 'apr': 4, 'may': 5, 'jun': 6, 'jul': 7, 'aug': 8, \
                 'sep': 9, 'oct': 10, 'nov': 11, 'dec': 12})
df['subscribe'] = df['subscribe'].map({'no': 0, 'yes': 1})

2.3 切分数据

# 将数据集重新划分为训练集和测试集   通过subscribe是不是空来判断
train = df[df['subscribe'].notnull()]
test = df[df['subscribe'].isnull()]

# 查看训练集中，标签为0和1的比例，可以看出0和1不均衡，0是1的6.6倍
train['subscribe'].value_counts()

得到

0.0    19548
1.0     2952
Name: subscribe, dtype: int64

2.4 分析数据

import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import warnings

warnings.filterwarnings('ignore')
%matplotlib inline
num_features = [x for x in train.columns if x not in cat_columns and x!='id']

fig = plt.figure(figsize=(80,60))

for i in range(len(num_features)):
    plt.subplot(7,2,i+1)
    sns.boxplot(train[num_features[i]])
    plt.ylabel(num_features[i], fontsize=36)
plt.show()

 存在离群点，对离群点进行处理

2.5 处理离群点

for colum in num_features:
    temp = train[colum]
    q1 = temp.quantile(0.25)
    q2 = temp.quantile(0.75)
    delta = (q2-q1) * 10
    train[colum] = np.clip(temp, q1-delta, q2+delta)
## 将超过10倍的值，进行处理

2.6 其他处理

进行数据均衡和特征选择，但是做完处理后都导致了分类效果变差，此处省略。但是把原码贴出来，供参考。

'''# 采用SMOTE进行过采样，虽然训练的效果好了，但是对于最终的分类效果反而降低了，此处先不采用过采样
from imblearn.over_sampling import SMOTE
from imblearn.over_sampling import ADASYN

#smo = SMOTE(random_state=0, k_neighbors=10)
adasyn = ADASYN()
X_smo, y_smo = adasyn.fit_resample(train.iloc[:,:-1], train.iloc[:,-1])
train_smo = pd.concat([X_smo, y_smo], axis=1)

train_smo['subscribe'].value_counts()'''

'''# 特征选择方法采用SelectFromModel，Model选择树模型
from sklearn.ensemble import ExtraTreesClassifier
from sklearn.feature_selection import SelectFromModel

# 提取出训练数据和标签
train_X = train.iloc[:,:-1]
train_y = train.iloc[:,-1]

# clf_ect是模型名，FeaSel为特征选择模型
clf_etc = ExtraTreesClassifier(n_estimators=50)
clf_etc = clf_etc.fit(train_X, train_y)
FeaSel = SelectFromModel(clf_etc, prefit=True)
train_sel = FeaSel.transform(train_X)
test_sel = FeaSel.transform(test.iloc[:,:-1])

# 提取特征名，并把特征名写回原始数据
train_new = pd.DataFrame(train_sel)
feature_idx = FeaSel.get_support() #提取选择的列名
train_new.columns = train_X.columns[feature_idx]  #将列名写回选择后的数据

train_new = pd.concat([train_new, train_y],axis=1)
test_new = pd.DataFrame(test_sel)
test_new.columns = train_X.columns[feature_idx]'''

此部门内容可能存在变量命名方面的问题。 

2.7 数据保存

train_new = train
test_new = test

# 将处理完的数据写回到train_new和test_new进行保存
train_new.to_csv('train_new.csv', index=False)
test_new.to_csv('test_new.csv', index=False)

3. 模型训练

3.1 导入包和数据 

from sklearn.model_selection import GridSearchCV
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.ensemble import AdaBoostClassifier
from xgboost import XGBRFClassifier
from lightgbm import LGBMClassifier
from sklearn.model_selection import cross_val_score
import time

clf_lr = LogisticRegression(random_state=0, solver='lbfgs', multi_class='multinomial')
clf_dt = DecisionTreeClassifier()
clf_rf = RandomForestClassifier()
clf_gb = GradientBoostingClassifier()
clf_adab = AdaBoostClassifier()
clf_xgbrf = XGBRFClassifier()
clf_lgb = LGBMClassifier()

from sklearn.model_selection import train_test_split
train_new = pd.read_csv('train_new.csv')
test_new = pd.read_csv('test_new.csv')
feature_columns = [col for col in train_new.columns if col not in ['subscribe']]
train_data = train_new[feature_columns]
target_data = train_new['subscribe']

3.2 模型调参 

from lightgbm import LGBMClassifier
from sklearn.metrics import classification_report
from sklearn.model_selection import GridSearchCV
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(train_data, target_data, test_size=0.2,shuffle=True, random_state=2023)
#X_test, X_valid, y_test, y_valid = train_test_split(X_test, y_test, test_size=0.5,shuffle=True,random_state=2023)

n_estimators = [300]
learning_rate = [0.02]#中0.2最优
subsample = [0.6]
colsample_bytree = [0.7]   ##在[0.5, 0.6, 0.7]中0.6最优
max_depth = [9, 11, 13] ##在[7, 9, 11, 13]中11最优
is_unbalance = [False]
early_stopping_rounds = [300]
num_boost_round = [5000]
metric = ['binary_logloss']
feature_fraction = [0.6, 0.75, 0.9]
bagging_fraction = [0.6, 0.75, 0.9]
bagging_freq = [2, 4, 5, 8]
lambda_l1 = [0, 0.1, 0.4, 0.5]
lambda_l2 =  [0, 10, 15, 35]
cat_smooth = [1, 10, 15, 20]


param = {'n_estimators':n_estimators,
         'learning_rate':learning_rate,
         'subsample':subsample,
         'colsample_bytree':colsample_bytree,
         'max_depth':max_depth,
         'is_unbalance':is_unbalance,
         'early_stopping_rounds':early_stopping_rounds,
         'num_boost_round':num_boost_round,
         'metric':metric,
         'feature_fraction':feature_fraction,
         'bagging_fraction':bagging_fraction,
         'lambda_l1':lambda_l1,
         'lambda_l2':lambda_l2,
         'cat_smooth':cat_smooth}

model = LGBMClassifier()

clf = GridSearchCV(model, param, cv=3, scoring='accuracy', verbose=1, n_jobs=-1)
clf.fit(X_train, y_train, eval_set=[(X_train, y_train),(X_test, y_test)])

print(clf.best_params_, clf.best_score_)

里面只有1个值的，是已经通过GridSearchCV找到的最优优值了，程序显示的是最后的6个参数的寻优，都放到一起训练时间太长了，所以选择分开寻找。

得到的结果：

Early stopping, best iteration is:
[287]	training's binary_logloss: 0.22302	valid_1's binary_logloss: 0.253303
{'bagging_fraction': 0.6, 'cat_smooth': 1, 'colsample_bytree': 0.7, 'early_stopping_rounds': 300, 'feature_fraction': 0.75, 'is_unbalance': False, 'lambda_l1': 0.4, 'lambda_l2': 10, 'learning_rate': 0.02, 'max_depth': 11, 'metric': 'binary_logloss', 'n_estimators': 300, 'num_boost_round': 5000, 'subsample': 0.6} 0.8853333333333334

3.3 预测结果

y_true, y_pred = y_test, clf.predict(X_test)
accuracy = accuracy_score(y_true,y_pred)
print(classification_report(y_true, y_pred))
print('Accuracy',accuracy)

结果

precision    recall  f1-score   support

         0.0       0.91      0.97      0.94      3933
         1.0       0.60      0.32      0.42       567

    accuracy                           0.89      4500
   macro avg       0.75      0.64      0.68      4500
weighted avg       0.87      0.89      0.87      4500

Accuracy 0.8875555555555555

查看混淆矩阵 

from sklearn import metrics
confusion_matrix_result = metrics.confusion_matrix(y_true, y_pred)
plt.figure(figsize=(8,6))
sns.heatmap(confusion_matrix_result, annot=True, cmap='Blues')
plt.xlabel('predict')
plt.ylabel('true')
plt.show()

 4. 输出结果

test_x = test[feature_columns]
pred_test = clf.predict(test_x)
result = pd.read_csv('./submission.csv')
subscribe_map ={1: 'yes', 0: 'no'}
result['subscribe'] = [subscribe_map[x] for x in pred_test]
result.to_csv('./baseline_lgb1.csv', index=False)
result['subscribe'].value_counts()

结果：

no     6987
yes     513
Name: subscribe, dtype: int64

5. 提交结果

 6. 总结

本人的方法只获得了0.9676的结果，希望您能在本人的程序基础上进行改进，以得到更佳的效果。如果有了更好的方法，欢迎在留言区告诉我，相互讨论。

改进的思路：

1. 数据处理方面，本人在进行数据均衡时，训练的效果很好，但是最终的效果较差，应该是数据过拟合了；另外在数据的离群点处理方面也可以做更进一步的考虑；

2.方法的改进，本人对比了lr, dt, rf, gb, adab, xgbrf, lgb最终lgb的效果最好，所以最终选择lgb进行调参，可以考虑采用多种方法的组合，进行训练；

3.在lgb的基础上进行调参，这个是最没有科技含量的。不过花时间应该会得到比我的结果更好的效果。