xgboost 毒蘑菇mushroom数据集分类

安装 xgboost

pip3 install xgboost

毒蘑菇数据集

毒蘑菇数据集的描述参考：
https://archive.ics.uci.edu/ml/datasets/Mushroom
毒蘑菇的特征描述如下

Attribute Information:

1. cap-shape: bell=b,conical=c,convex=x,flat=f, knobbed=k,sunken=s 
2. cap-surface: fibrous=f,grooves=g,scaly=y,smooth=s 
3. cap-color: brown=n,buff=b,cinnamon=c,gray=g,green=r, pink=p,purple=u,red=e,white=w,yellow=y 
4. bruises?: bruises=t,no=f 
5. odor: almond=a,anise=l,creosote=c,fishy=y,foul=f, musty=m,none=n,pungent=p,spicy=s 
6. gill-attachment: attached=a,descending=d,free=f,notched=n 
7. gill-spacing: close=c,crowded=w,distant=d 
8. gill-size: broad=b,narrow=n 
9. gill-color: black=k,brown=n,buff=b,chocolate=h,gray=g, green=r,orange=o,pink=p,purple=u,red=e, white=w,yellow=y 
10. stalk-shape: enlarging=e,tapering=t 
11. stalk-root: bulbous=b,club=c,cup=u,equal=e, rhizomorphs=z,rooted=r,missing=? 
12. stalk-surface-above-ring: fibrous=f,scaly=y,silky=k,smooth=s 
13. stalk-surface-below-ring: fibrous=f,scaly=y,silky=k,smooth=s 
14. stalk-color-above-ring: brown=n,buff=b,cinnamon=c,gray=g,orange=o, pink=p,red=e,white=w,yellow=y 
15. stalk-color-below-ring: brown=n,buff=b,cinnamon=c,gray=g,orange=o, pink=p,red=e,white=w,yellow=y 
16. veil-type: partial=p,universal=u 
17. veil-color: brown=n,orange=o,white=w,yellow=y 
18. ring-number: none=n,one=o,two=t 
19. ring-type: cobwebby=c,evanescent=e,flaring=f,large=l, none=n,pendant=p,sheathing=s,zone=z 
20. spore-print-color: black=k,brown=n,buff=b,chocolate=h,green=r, orange=o,purple=u,white=w,yellow=y 
21. population: abundant=a,clustered=c,numerous=n, scattered=s,several=v,solitary=y 
22. habitat: grasses=g,leaves=l,meadows=m,paths=p, urban=u,waste=w,woods=d

mushroom数据集原有的格式是：

p,x,s,n,t,p,f,c,n,k,e,e,s,s,w,w,p,w,o,p,k,s,u
e,x,s,y,t,a,f,c,b,k,e,c,s,s,w,w,p,w,o,p,n,n,g

第一列的对应关系是：无毒 edible=e, 有毒 poisonous=p，后面的列是特征。
显然，上述特征值都是categorical 特征，跟上面22个特征描述一一对应。对于这种categorical feature，一般都要进行onehot编码，可以借助sklearn的DictVectorizer或者自己编写onehot，然后按照xgboost的输入格式 ：
category feature_id:feature_value...

例如xgboost tutorial里面内置的mushroom数据集：

1 3:1 10:1 11:1 21:1 30:1 34:1 36:1 40:1 41:1 53:1 58:1 65:1 69:1 77:1 86:1 88:1 92:1 95:1 102:1 105:1 117:1 124:1
0 3:1 10:1 20:1 21:1 23:1 34:1 36:1 39:1 41:1 53:1 56:1 65:1 69:1 77:1 86:1 88:1 92:1 95:1 102:1 106:1 116:1 120:1

由于xgboost的repo已经内置了mushroom的数据集，并且格式已经调整好，所以就直接用吧（懒～）

分类验证

先把 https://github.com/dmlc/xgboost 这个repo clone下来，demo/data里面有处理好的mushroom数据集
下面跑一下交叉验证的代码

#!/usr/bin/python
import numpy as np
import xgboost as xgb
import json

### load data in do training
dtrain = xgb.DMatrix('data/agaricus.txt.train')
param = {'max_depth':2, 'eta':1, 'silent':1, 'objective':'binary:logistic'}
num_round = 2

print('running cross validation')
# do cross validation, this will print result out as
# [iteration]  metric_name:mean_value+std_value
# std_value is standard deviation of the metric
xgb.cv(param, dtrain, num_round, nfold=5,
       metrics={'error'}, seed=0,
       callbacks=[xgb.callback.print_evaluation(show_stdv=True)])

print('running cross validation, disable standard deviation display')
# do cross validation, this will print result out as
# [iteration]  metric_name:mean_value
res = xgb.cv(param, dtrain, num_boost_round=10, nfold=5,
             metrics={'error',"auc","rmse"}, seed=0,
             callbacks=[xgb.callback.print_evaluation(show_stdv=False),
                        xgb.callback.early_stop(3)])
print(json.dumps(res, indent=4))

评价的几个维度，分别是error, auc, rmse，输入如下：

[22:46:13] 6513x127 matrix with 143286 entries loaded from ../data/agaricus.txt.train
running cross validation
[0]	train-error:0.0506682+0.009201	test-error:0.0557316+0.0158887
[1]	train-error:0.0213034+0.00205561	test-error:0.0211884+0.00365323
running cross validation, disable standard deviation display
[0]	train-auc:0.962827	train-error:0.0506682	train-rmse:0.230558	test-auc:0.960993	test-error:0.0557316	test-rmse:0.234907
Multiple eval metrics have been passed: 'test-rmse' will be used for early stopping.

Will train until test-rmse hasn't improved in 3 rounds.
[1]	train-auc:0.984829	train-error:0.0213034	train-rmse:0.159816	test-auc:0.984753	test-error:0.0211884	test-rmse:0.159676
[2]	train-auc:0.99763	train-error:0.0099418	train-rmse:0.111782	test-auc:0.997216	test-error:0.0099786	test-rmse:0.113232
[3]	train-auc:0.998845	train-error:0.0141256	train-rmse:0.104002	test-auc:0.998575	test-error:0.0144336	test-rmse:0.105863
[4]	train-auc:0.999404	train-error:0.0059878	train-rmse:0.078452	test-auc:0.999001	test-error:0.0062948	test-rmse:0.0814638
[5]	train-auc:0.999571	train-error:0.0020344	train-rmse:0.0554116	test-auc:0.999236	test-error:0.0016886	test-rmse:0.0549618
[6]	train-auc:0.999643	train-error:0.0012284	train-rmse:0.0442974	test-auc:0.999389	test-error:0.001228	test-rmse:0.0447266
[7]	train-auc:0.999736	train-error:0.0012284	train-rmse:0.0409082	test-auc:0.999535	test-error:0.001228	test-rmse:0.0408704
[8]	train-auc:0.999967	train-error:0.0009212	train-rmse:0.0325856	test-auc:0.99992	test-error:0.001228	test-rmse:0.0378632
[9]	train-auc:0.999982	train-error:0.0006142	train-rmse:0.0305786	test-auc:0.999959	test-error:0.001228	test-rmse:0.0355032
{
    "train-auc-mean": [
        0.9628270000000001,
        0.9848285999999999,
        0.9976303999999999,
        0.9988453999999999,
        0.9994036000000002,
        0.9995708000000001,
        0.9996430000000001,
        0.9997361999999999,
        0.9999673999999998,
        0.9999824
    ],
    "train-auc-std": [
        0.008448006628785306,
        0.006803294807664875,
        0.0009387171245907945,
        0.0003538839357755705,
        0.0002618454506001579,
        0.00025375531521528084,
        0.00020217319307960657,
        0.00020191027710344742,
        3.975726348731663e-05,
        2.0401960690100178e-05
    ],
    "train-error-mean": [
        0.0506682,
        0.0213034,
        0.009941799999999999,
        0.014125599999999999,
        0.0059878,
        0.0020344,
        0.0012284,
        0.0012284,
        0.0009212,
        0.0006142
    ],
    "train-error-std": [
        0.009200997193782855,
        0.0020556122786167634,
        0.006076479256938181,
        0.0017057689878761427,
        0.0018779069625516596,
        0.001469605198684327,
        0.00026026494193417596,
        0.00026026494193417596,
        0.0005061973528180487,
        0.000506318634853587
    ],
    "train-rmse-mean": [
        0.2305582,
        0.1598156,
        0.11178239999999999,
        0.104002,
        0.07845200000000001,
        0.05541159999999999,
        0.0442974,
        0.04090819999999999,
        0.0325856,
        0.0305786
    ],
    "train-rmse-std": [
        0.0037922719522734682,
        0.01125541159798254,
        0.002488777016930202,
        0.004049203625405865,
        0.0013676392799272755,
        0.0037808676041353262,
        0.0029096398127603355,
        0.002357843285716845,
        0.006475110519520113,
        0.005710078619423729
    ],
    "test-auc-mean": [
        0.9609932000000001,
        0.9847534,
        0.9972156,
        0.9985745999999999,
        0.9990005999999999,
        0.9992364,
        0.9993892000000001,
        0.9995353999999999,
        0.9999202,
        0.9999590000000002
    ],
    "test-auc-std": [
        0.01004388481415432,
        0.0073321434137637925,
        0.001719973674217141,
        0.0008885571675474929,
        0.0007358270448957148,
        0.0007796844489920071,
        0.0006559303011753682,
        0.00040290524940736174,
        9.667760857612425e-05,
        4.69212105555625e-05
    ],
    "test-error-mean": [
        0.055731600000000006,
        0.021188400000000003,
        0.009978599999999999,
        0.0144336,
        0.006294800000000001,
        0.0016885999999999997,
        0.001228,
        0.001228,
        0.001228,
        0.001228
    ],
    "test-error-std": [
        0.015888666194492227,
        0.0036532266614597024,
        0.004827953421482027,
        0.003517125508138713,
        0.0031231752688570006,
        0.0005741844999649501,
        0.0010409403441119958,
        0.0010409403441119958,
        0.0010409403441119958,
        0.0010409403441119958
    ],
    "test-rmse-mean": [
        0.2349068,
        0.1596758,
        0.113232,
        0.10586340000000001,
        0.08146379999999999,
        0.054961800000000005,
        0.04472659999999999,
        0.040870399999999994,
        0.0378632,
        0.0355032
    ],
    "test-rmse-std": [
        0.009347684363520205,
        0.013838324138420807,
        0.006189637210693368,
        0.008622184470306812,
        0.007872053592297248,
        0.009846015506792583,
        0.010218353460318349,
        0.0130493180756697,
        0.013843686595701305,
        0.013998153155327313
    ]
}

看rmse效果，说明xgboost的分类效果还是不错的。