集成学习(下)——Task6 分类模型

机器学习专题

机器学习三要素
模式识别 贝叶斯推导
机器学习训练的套路
考试前千万不要背书
你以为好好学习就可以考好了？
分类模型

(https://blog.csdn.net/weixin_43913783/article/details/114852227?spm=1001.2014.3001.5501)

进度条

基于fetch_lfw_people人脸数据的分类实战¹

from sklearn.datasets import fetch_lfw_people
faces = fetch_lfw_people(min_faces_per_person = 60)
print(faces.target_names)
print(faces.images.shape)

['Ariel Sharon' 'Colin Powell' 'Donald Rumsfeld' 'George W Bush'
 'Gerhard Schroeder' 'Hugo Chavez' 'Junichiro Koizumi' 'Tony Blair']
(1348, 62, 47)

简单查看数据

import matplotlib.pyplot as plt
fig, ax = plt.subplots(3, 5)
for i, axi in enumerate(ax.flat):
    axi.imshow(faces.images[i], cmap = 'gray')
    axi.set(xticks = [], yticks = [],
            xlabel = faces.target_names[faces.target[i]])

降维处理

from sklearn.svm import SVC
from sklearn.decomposition import PCA
from sklearn.pipeline import make_pipeline #make_pipelin是Pipeline类的简单实现

pca  = PCA(n_components = 150, whiten = True) #因为SVM要求数据必须预处理，所以whiten=True
svc = SVC(kernel = 'rbf')
model = make_pipeline(pca, svc)

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(faces.data, faces.target)

from sklearn.model_selection import GridSearchCV
param = {'svc__C': [1, 5, 10, 50, 75, 90],
         'svc__gamma': [0.0001, 0.0005, 0.001, 0.005]}
grid = GridSearchCV(model, param, cv = 5)
grid.fit(X_train, y_train)

GridSearchCV(cv=5,
             estimator=Pipeline(steps=[('pca',
                                        PCA(n_components=150, whiten=True)),
                                       ('svc', SVC())]),
             param_grid={'svc__C': [1, 5, 10, 50, 75, 90],
                         'svc__gamma': [0.0001, 0.0005, 0.001, 0.005]})

构建管道(make_pipeline) 知识笔记

为了简化构建变换和模型链的过程，Scikit-Learn提供了pipeline类，可以将多个处理步骤合并为单个Scikit-Learn估计器。pipeline类本身具有fit、predict和score方法，其行为与Scikit-Learn中的其他模型相同。

一、使用Pipeline类来表示在使用MinMaxScaler缩放数据之后再训练一个SVM的工作流程

from sklearn.pipeline import Pipeline
pipe = Pipeline([("scaler",MinMaxScaler()),("svm",SVC())])
pip.fit(X_train,y_train)
pip.score(X_test,y_test)

一个由步骤列表组成的管道对象，每个步骤都是一个元组，包含一个名称（自定义）和一个估计器的实例。首先对第一个步骤（缩放器）调用fit，然后使用该缩放器对训练数据进行变换，最后利用缩放后的数据来拟合SVM。利用管道，我们减少了“预处理+分类”过程所需要的代码量，而且，使用管道的主要优点在于，我们可以在cross_val_score或GridSearchCV中使用这个估计器。

二、在网格搜索中使用管道

param_grid = {'svm__C':[0.001,0.01,0.1,1,10,100],'svm__gamma':[0.001,0.01,0.1,1,10,100]}
grid = GridSearchCV(pipe,param_grid=param_grid,cv=5)
grid_search.fit(X_train,y_train)
print("Test set score:{:.2f}".format(grid_search.score(X_test,y_test)))   

我们需要为每个参数（C和gamma）指定它在管道中所属的步骤。为管道定义参数网格的语法是为每个参数指定步骤名称，后面加上__（双下划线），然后是参数名称（svm__C和svm_gamma）。

三、用make_pipeline创建管道

用Pipeline类构建管道时语法有点麻烦，我们通常不需要为每一个步骤提供用户指定的名称，这种情况下，就可以用make_pipeline函数创建管道，它可以为我们创建管道并根据每个步骤所属的类为其自动命名。

StandardScaler和MinMaxScaler来源于sklearn.preprocessing

# 方式2：随机网格搜索RandomizedSearchCV()
from sklearn.model_selection import RandomizedSearchCV
from sklearn.svm import SVC
import time

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.preprocessing import MinMaxScaler

start_time = time.time()
#pipe_svc = make_pipeline(MinMaxScaler(),SVC(random_state=1))
pipe_svc = make_pipeline(StandardScaler(),SVC(random_state=1))
param_range = [0.0001,0.001,0.01,0.1,1.0,10.0,100.0,1000.0]
param_grid = [{'svc__C':param_range,'svc__kernel':['linear']},{'svc__C':param_range,'svc__gamma':param_range,'svc__kernel':['rbf']}]
# param_grid = [{'svc__C':param_range,'svc__kernel':['linear','rbf'],'svc__gamma':param_range}]
gs = RandomizedSearchCV(estimator=pipe_svc, param_distributions=param_grid,scoring='accuracy',cv=10,n_jobs=-1)
gs = gs.fit(X_train, y_train)
end_time = time.time()
print("随机网格搜索经历时间：%.3f S" % float(end_time-start_time))
print(gs.best_score_)
print(gs.best_params_)

随机网格搜索经历时间：360.049 S
0.8219180741603573
{'svc__kernel': 'linear', 'svc__C': 0.1}

print(grid.best_params_)
print("最高分{:.3f}".format(grid.best_score_))

{'svc__C': 75, 'svc__gamma': 0.005}
最高分0.829

y_pre = grid.predict(X_test)
fig, ax = plt.subplots(4,6)
for i, axi in enumerate(ax.flat):
    axi.imshow(X_test[i].reshape(62,47), cmap = 'gray')
    axi.set(xticks = [], yticks = [])
    axi.set_xlabel(faces.target_names[y_pre[i]],
                   color = 'k' if y_pre[i] == y_test[i] else 'r')

from sklearn.metrics import classification_report
print(classification_report(y_test, y_pre))

              precision    recall  f1-score   support

           0       1.00      0.71      0.83        21
           1       0.81      0.92      0.86        60
           2       0.96      0.76      0.85        33
           3       0.83      0.95      0.89       131
           4       0.80      0.74      0.77        27
           5       1.00      0.53      0.69        19
           6       1.00      0.88      0.93        16
           7       0.79      0.77      0.78        30

    accuracy                           0.85       337
   macro avg       0.90      0.78      0.83       337
weighted avg       0.86      0.85      0.85       337

---

1. Datawhale 组队学习——集成学习 ↩︎