Kaggle 题目 nu-cs6220-assignment-1

最新推荐文章于 2024-11-08 17:49:58 发布

ZackFairT

最新推荐文章于 2024-11-08 17:49:58 发布

阅读量89

点赞数

文章标签： mysql 数据库

本文链接：https://blog.csdn.net/tsjjjjj/article/details/133007772

版权

Kaggle题目 nu-cs6220-assignment-1

题目地址如下：

https://www.kaggle.com/c/nu-cs6220-assignment-1/overview

这是个二分类任务，需要预测一个人的收入，分为两类：收入大于50K，或是小于50K。

1. 查看数据结构

下载数据后，先大致了解数据：

raw_data = load_data('nu-cs/training.txt')

raw_data.head()

可以看到没有header，根据题目对数据的说明，给它们分配header：

header = ['age', 'workclass', 'fnlwgt', 'education', 'education-num',
        'marital-status', 'occupation', 'relationship', 'race', 'sex',
        'capital-gain', 'capital-loss', 'hours-per-week', 'native-country', 'salary']

raw_data.columns = header
raw_data.head()

在这个问题中，label为’salary’，这里它是一个离散变量，可以看到其中一个值是 ‘ <=50K’。进一步查看一下这个label包含的离散值：

raw_data['salary'].value_counts()

可以看到仅包含两类，且无缺失值或异常值。

继续查看数据集描述：

raw_data.info()

一共15个特征，6个为连续型，9个为离散型。数据条目为32560，每个特征均包含32560，但这个并不能说明数据集中没有缺失值，根据题目描述，缺失值已由 ? 代替。

2. 数值型特征

对于数值型特征，看一下统计数据：

raw_data.describe()

以及直方图：

结合这两组信息，我们可以看到有几点需要注意的地方：

Capital-gain与capital-loss 大部分的值都是0，但是最大值却非常大，导致方差较大。
有些直方图是长尾分布，可以尝试将它们的分布转为钟型分布
部分特征需要进行分箱处理

下面先依次处理连续型变量。

2.1. Age特征

首先对于age特征，对它进行分箱并查看它们的相关性：

raw_data['age_band'] = pd.cut(raw_data['age'], 5)

raw_data[['age_band', 'salary']].groupby(['age_band'], as_index=False).mean().sort_values(by='age_band', ascending=True)

然后根据此分段，使用有序值替换 age 的值：

raw_data.loc[raw_data['age'] <= 31.6, 'age'] = 0
raw_data.loc[(raw_data['age'] <= 46.2) & (raw_data['age'] > 31.6), 'age'] = 1
raw_data.loc[(raw_data['age'] <= 60.8) & (raw_data['age'] > 46.2), 'age'] = 2
raw_data.loc[(raw_data['age'] <= 75.4) & (raw_data['age'] > 60.8), 'age'] = 3
raw_data.loc[(raw_data['age'] <= 90.0) & (raw_data['age'] > 75.4), 'age'] = 4

检查结果：

raw_data['age'].value_counts()
1    12210
0    11460
2     6558
3     2091
4      241

Name: age, dtype: int64

最后丢弃age_band 特征：

raw_data = raw_data.drop(['age_band'], axis=1)

raw_data.head()

2.2. fnlwgt特征

这个特征的问题在于：数值范围和方差都非常的大。

首先看它的直方图：

raw_data['fnlwgt'].hist(bins=100)

可以看到取值范围非常广，且类长尾分布。我们对它取对数，然后再观察它的直方图：

import numpy as np

raw_data['log_fnlwgt'] = raw_data['fnlwgt'].apply(np.log)

raw_data[['log_fnlwgt','fnlwgt']].hist(bins=100)

可以看到取对数后更接近钟型分布。最后，丢弃 log_fnlwgt，并直接在fnlwgt 上做变换：

raw_data.drop(['log_fnlwgt'], axis=1)

raw_data['fnlwgt'] = raw_data['fnlwgt'].apply(np.log)

2.3. Education-num

对于 education-num，它的取值虽然是数值型，但训练集中为有限集：

raw_data['education-num'].value_counts()
9     10501
10     7291
13     5354
14     1723
11     1382
7      1175
12     1067
6       933
4       646
15      576
5       514
8       433
16      413
3       333
2       168
1        51
Name: education-num, dtype: int64

同样对它使用区间量化，同 age 特征。过程在此不赘述，处理后的结果：

raw_data['education-num'].value_counts()
2    18225
3     7803
4     2712
1     2622
0     1198
Name: education-num, dtype: int64

2.4. capital-gain 与 capital-loss

raw_data[['capital-gain', 'capital-loss']].hist()

这两个特征的特点是：大部分值都为0，少部分值特别大。对这两个特征，采用二值化处理：

raw_data[['capital-gain', 'capital-loss']] = (raw_data[['capital-gain', 'capital-loss']] > 0) * 1

处理后结果为：

raw_data['capital-gain'].value_counts()
0    29849
1     2711
Name: capital-gain, dtype: int64

raw_data['capital-loss'].value_counts()
0    31041
1     1519
Name: capital-loss, dtype: int64

2.5. hours-per-week

此特征的图像类似为钟型分布，可以直接做标准化处理，或是做分桶处理均可，在此做了分桶处理，过程不追溯。

3. 离散特征

对于离散型特征，我们会用One-Hot 编码处理。首先我们清理缺失值：

workclass中存在 1836 条缺失值：

raw_data['workclass'].value_counts()
Private             22696
 Self-emp-not-inc     2541
 Local-gov            2093
 ?                    1836
 State-gov            1297
 Self-emp-inc         1116
 Federal-gov           960
 Without-pay            14
 Never-worked            7
Name: workclass, dtype: int64

occupation 中存在 1843 条缺失值：

raw_data['occupation'].value_counts()
 Prof-specialty       4140
 Craft-repair         4099
 Exec-managerial      4066
 Adm-clerical         3769
 Sales                3650
 Other-service        3295
 Machine-op-inspct    2002
 ?                    1843
 Transport-moving     1597
 Handlers-cleaners    1370
 Farming-fishing       994
 Tech-support          928
 Protective-serv       649
 Priv-house-serv       149
 Armed-Forces            9
Name: occupation, dtype: int64

native-country 中存在583 条缺失值：

raw_data['native-country'].value_counts()
 United-States                 29169
 Mexico                          643
 ?                               583
 Philippines                     198
  …

对于这些缺失值，我们简单地使用众数来填充这个缺失值：

freq_workclass = raw_data.workclass.mode()[0]
raw_data.loc[(raw_data['workclass'] == ' ?'), 'workclass'] = freq_workclass

freq_occupation = raw_data.occupation.mode()[0]
raw_data.loc[(raw_data['occupation'] == ' ?'), 'occupation'] = freq_workclass

freq_nativecountry = raw_data['native-country'].mode()[0]
raw_data.loc[(raw_data['native-country'] == ' ?'), 'native-country'] = freq_nativecountry

补全缺失值后，我们可以对它们应用one-hot 编码。不过对于native-country 特征，里面包含的离散值类别过多，若是使用 one-hot 编码，则势必会造成特征维度大大增加。这里我们用更少的特征去对它们进行替换：

raw_data.loc[raw_data['native-country'] == ' Scotland', 'native-country'] = 'UK'
raw_data.loc[raw_data['native-country'] == ' United-States', 'native-country'] = 'US'
raw_data.loc[raw_data['native-country'] == ' Mexico', 'native-country'] = 'South-America'
raw_data.loc[raw_data['native-country'] == ' Jamaica', 'native-country'] = 'South-America'
raw_data.loc[raw_data['native-country'] == ' Philippines', 'native-country'] = 'Asia'
raw_data.loc[raw_data['native-country'] == ' Germany', 'native-country'] = 'Euro'
raw_data.loc[raw_data['native-country'] == ' Canada', 'native-country'] = 'North-America'
raw_data.loc[raw_data['native-country'] == ' Puerto-Rico', 'native-country'] = 'South-America'
raw_data.loc[raw_data['native-country'] == ' El-Salvador', 'native-country'] = 'South-America'
raw_data.loc[raw_data['native-country'] == ' India', 'native-country'] = 'Asia'
raw_data.loc[raw_data['native-country'] == ' Cuba', 'native-country'] = 'South-America'
raw_data.loc[raw_data['native-country'] == ' England', 'native-country'] = 'UK'
raw_data.loc[raw_data['native-country'] == ' Italy', 'native-country'] = 'Euro'
raw_data.loc[raw_data['native-country'] == ' Dominican-Republic', 'native-country'] = 'South-America'
raw_data.loc[raw_data['native-country'] == ' Vietnam', 'native-country'] = 'Asia'
raw_data.loc[raw_data['native-country'] == ' Guatemala', 'native-country'] = 'South-America'
raw_data.loc[raw_data['native-country'] == ' Poland', 'native-country'] = 'Euro'
raw_data.loc[raw_data['native-country'] == ' Columbia', 'native-country'] = 'South-America'
raw_data.loc[raw_data['native-country'] == ' Haiti', 'native-country'] = 'South-America'
raw_data.loc[raw_data['native-country'] == ' Portugal', 'native-country'] = 'Euro'
raw_data.loc[raw_data['native-country'] == ' Greece', 'native-country'] = 'Euro'
raw_data.loc[raw_data['native-country'] == ' France', 'native-country'] = 'Euro'
raw_data.loc[raw_data['native-country'] == ' Ireland', 'native-country'] = 'Euro'
raw_data.loc[raw_data['native-country'] == ' Holand-Netherlands', 'native-country'] = 'Euro'
raw_data.loc[raw_data['native-country'] == ' China', 'native-country'] = 'Asia'
raw_data.loc[raw_data['native-country'] == ' Japan', 'native-country'] = 'Asia'
raw_data.loc[raw_data['native-country'] == ' Taiwan', 'native-country'] = 'Asia'
raw_data.loc[raw_data['native-country'] == ' Hong', 'native-country'] = 'Asia'
raw_data.loc[raw_data['native-country'] == ' Nicaragua', 'native-country'] = 'South-America'
raw_data.loc[raw_data['native-country'] == ' Peru', 'native-country'] = 'South-America'
raw_data.loc[raw_data['native-country'] == ' Ecuador', 'native-country'] = 'South-America'
raw_data.loc[raw_data['native-country'] == ' Cambodia', 'native-country'] = 'Asia'
raw_data.loc[raw_data['native-country'] == ' Thailand', 'native-country'] = 'Asia'
raw_data.loc[raw_data['native-country'] == ' Laos', 'native-country'] = 'Asia'
raw_data.loc[raw_data['native-country'] == ' Trinadad&Tobago', 'native-country'] = 'South-America'
raw_data.loc[raw_data['native-country'] == ' Yugoslavia', 'native-country'] = 'Euro'
raw_data.loc[raw_data['native-country'] == ' Honduras', 'native-country'] = 'South-America'
raw_data.loc[raw_data['native-country'] == ' Hungary', 'native-country'] = 'Euro'
raw_data.loc[raw_data['native-country'] == ' Iran', 'native-country'] = 'Middle-East'
raw_data.loc[raw_data['native-country'] == ' South', 'native-country'] = 'South-America'

raw_data['native-country'].value_counts()
US                             29752
South-America                   1481
Asia                             628
Euro                             419
North-America                    121
UK                               102
Middle-East                       43
 Outlying-US(Guam-USVI-etc)       14
Name: native-country, dtype: int64

另一个可以进一步处理的特征是workclass，可以看到 workclass里的类别为：

Raw_data['workclass'].value_counts()
 Private             24532
 Self-emp-not-inc     2541
 Local-gov            2093
 State-gov            1297
 Self-emp-inc         1116
 Federal-gov           960
 Without-pay            14
 Never-worked            7
Name: workclass, dtype: int64

其中 Without-pay 与 Nerver-worked 数量都比较少，也意思接近，我们将它作为一个类别处理：

def change_workclass(df):
    df.loc[df['workclass'] == ' Without-pay', 'workclass'] = 'No-pay'
    df.loc[df['workclass'] == ' Never-worked', 'workclass'] = 'No-pay'

4. 数据中心化、标准化以及One-Hot编码

在连续性变量与离散型变量均处理完毕后，将特征数据与label数据分离：

def get_data_label(df, label):
    dataset = df.drop(label, axis=1)
    labels = df[label].copy()
    return dataset, labels

dataset, labels = get_data_label(raw_data, 'salary')

然后分别对数值型做中心化与标准化，离散值做one-hot编码：

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.impute import SimpleImputer

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.impute import SimpleImputer
from sklearn.compose import ColumnTransformer

num_pipeline = Pipeline([
    ('imputer', SimpleImputer(strategy='median')),
    ('std_scaler', StandardScaler()),
])


full_pipeline = ColumnTransformer([
    ('num', num_pipeline, num_attributes),
    ('cat', OneHotEncoder(), cat_attributes),
])

nu_cs_prepared = full_pipeline.fit_transform(dataset)

模型训练

首先我们用sk-learn提供的几个模型训练：

from sklearn.model_selection import cross_val_score
cross_val_score(tree_clf, nu_cs_prepared, labels, cv=10, scoring='accuracy')
array([0.77955173, 0.78194103, 0.78961916, 0.77610565, 0.79299754,
       0.7779484 , 0.78808354, 0.79207617, 0.79391892, 0.77695853])

from sklearn.svm import LinearSVC
svm_clf = LinearSVC(C=3, loss="hinge")
cross_val_score(svm_clf, nu_cs_prepared, labels, cv=10, scoring='accuracy')
array([0.83911575, 0.83814496, 0.84520885, 0.82800983, 0.84029484,
       0.84459459, 0.83630221, 0.84735872, 0.84490172, 0.83870968])

# logistic regression
logreg = LogisticRegression()
cross_val_score(logreg, nu_cs_prepared, labels, cv=10, scoring='accuracy')
array([0.84280012, 0.84029484, 0.84981572, 0.83169533, 0.84398034,
       0.84797297, 0.84029484, 0.8470516 , 0.85104423, 0.84423963])

from sklearn.ensemble import RandomForestClassifier
rnd_clf = RandomForestClassifier(n_estimators=500, max_leaf_nodes=16, n_jobs=-1)
cross_val_score(rnd_clf, nu_cs_prepared, labels, cv=10, scoring='accuracy')
array([0.82222905, 0.82493857, 0.83015971, 0.82340295, 0.82831695,
       0.82985258, 0.82186732, 0.83476658, 0.8252457 , 0.82519201])

可以看到表现最好的是SVM和LR。下面选择SVM，进行超参数搜索：

from sklearn.model_selection import GridSearchCV

param_grid = [
    {'C':[1, 3, 10, 30], 'loss':['hinge'], 'dual':[True]}
]

svm_clf = LinearSVC()
grid_search = GridSearchCV(svm_clf, param_grid, cv=5,
                           scoring='accuracy',
                           return_train_score=True)

grid_search.fit(nu_cs_prepared, labels)

grid_search.best_estimator_
LinearSVC(C=30, class_weight=None, dual=True, fit_intercept=True,
     intercept_scaling=1, loss='hinge', max_iter=1000, multi_class='ovr',
     penalty='l2', random_state=None, tol=0.0001, verbose=0)
grid_search.best_score_
0.832063882063882

可以看到最好的模型C=30(所以还可以往上调整C进一步搜索)，此时的准确率为83%，但仍比不上 LR 的准确率。

再试试对随机森林的超参数搜索：

from sklearn.model_selection import GridSearchCV

param_grid = [
    {'n_estimators':[3, 10, 30], 'max_features':[2, 4, 6, 8]},
    {'bootstrap':[False], 'n_estimators':[3, 10], 'max_features':[2, 3, 4]},
]

forest_clf = RandomForestClassifier()
grid_search = GridSearchCV(forest_clf, param_grid, cv=5,
                           scoring='accuracy',
                           return_train_score=True)

grid_search.fit(nu_cs_prepared, labels)

表现最好的参数为：

grid_search.best_params_
{'max_features': 8, 'n_estimators': 30}

最高分为：

0.8196253071253071

效果仍逊色于LR 的平均0.84 左右，下一章再试试 sagemaker 对模型进行训练。

ZackFairT

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
复制链接

分享到 QQ

分享到新浪微博

扫一扫