python与机器学习（六）——支持向量机(SVM) && 多层感知机(MLP)

在这次实验中，我们将尝试提取基本的图像特征并利用支持向量机或多层感知机算法对提取的特征进行图像分类。

导入：

import numpy as np
import matplotlib
from scipy.ndimage import uniform_filter

数据加载：

# 读取提供的cifar10-mini数据集，
data = np.load('cifar10-mini.npz')

X_train= data['X_train']
X_val= data['X_val']
X_test= data['X_test']
y_train= data['y_train']
y_val= data['y_val']
y_test= data['y_test']

print(X_train.shape)
print(X_val.shape)
print(X_test.shape)

运行结果如下：

(5000, 32, 32, 3)
(500, 32, 32, 3)
(500, 32, 32, 3)

提取图像特征：

方向梯度直方图 HOG (Histogram of Oriented Gridients)特征检测算法，最早是由法国研究员Dalal等在CVPR-2005上提出来的，一种解决人体目标检测的图像描述子，是一种用于表征图像局部梯度方向和梯度强度分布特性的描述符。其主要思想是：在边缘具体位置未知的情况下，边缘方向的分布也可以很好的表示图像中物体的外形轮廓，但会忽略掉颜色信息。特征维度是144维。

hog_feature是接收一张图像然后返回这张图像的特征向量。你可以使用这个函数提取所有图像的特征并将其存入 X_train_feats, X_val_feats, X_test_feats 这三个变量中（他们分别代表训练集、验证集和测试集的特征）。

def hog_feature(im):
    # convert rgb to grayscale if needed
    if im.ndim == 3:
        image = np.dot(im[...,:3], [0.299, 0.587, 0.144])
    else:
        image = np.atleast_2d(im)

    sx, sy = image.shape # image size
    orientations = 9 # number of gradient bins
    cx, cy = (8, 8) # pixels per cell

    gx = np.zeros(image.shape)
    gy = np.zeros(image.shape)
    gx[:, :-1] = np.diff(image, n=1, axis=1) # compute gradient on x-direction
    gy[:-1, :] = np.diff(image, n=1, axis=0) # compute gradient on y-direction
    grad_mag = np.sqrt(gx ** 2 + gy ** 2) # gradient magnitude
    grad_ori = np.arctan2(gy, (gx + 1e-15)) * (180 / np.pi) + 90 # gradient orientation

    n_cellsx = int(np.floor(sx / cx))  # number of cells in x
    n_cellsy = int(np.floor(sy / cy))  # number of cells in y
    # compute orientations integral images
    orientation_histogram = np.zeros((n_cellsx, n_cellsy, orientations))
    for i in range(orientations):
        # create new integral image for this orientation
        # isolate orientations in this range
        temp_ori = np.where(grad_ori < 180 / orientations * (i + 1),
                            grad_ori, 0)
        temp_ori = np.where(grad_ori >= 180 / orientations * i,
                            temp_ori, 0)
        # select magnitudes for those orientations
        cond2 = temp_ori > 0
        temp_mag = np.where(cond2, grad_mag, 0)
        orientation_histogram[:,:,i] = uniform_filter(temp_mag, size=(cx, cy))[int(cx/2)::cx, int(cy/2)::cy].T

    return orientation_histogram.ravel()

X_train_feats = [hog_feature(X_train[i]) for i in range(X_train.shape[0])]
X_val_feats = [hog_feature(X_val[i]) for i in range(X_val.shape[0])]
X_test_feats = [hog_feature(X_test[i]) for i in range(X_test.shape[0])]


# 预处理: 减去均值
mean_feat = np.mean(X_train_feats, axis=0, keepdims=True)
X_train_feats -= mean_feat
X_val_feats -= mean_feat
X_test_feats -= mean_feat

# 预处理: 除以标准差
std_feat = np.std(X_train_feats, axis=0, keepdims=True)
X_train_feats /= std_feat
X_val_feats /= std_feat
X_test_feats /= std_feat

print(X_train_feats.shape)
print(X_val_feats.shape)
print(X_test_feats.shape)

运行结果如下：

(5000, 144)
(500, 144)
(500, 144)

使用支持向量机(SVM)进行分类：

使用不同的核函数作为SVM模型的参数，选择验证集上准确率最好的SVM核函数，将训练集和验证集组合训练上面选择的核函数SVM模型，在测试集上评估正确率，并查看各类别的查准率、查全率等指标。

from sklearn.svm import SVC
from sklearn.metrics import classification_report

# 训练集、验证集特征及标签组合
X_train_val_feats = None
y_train_val = None

使用多层感知机(MLP)进行分类：

自行定义多层感知机的batch_size, hidden_layer_sizes, solver, learning_rate_init, max_iter, activation等超参数进行模型训练，将训练集和验证集组合训练上面选择的超参数MLP模型，在测试集上评估正确率，并查看各类别的查准率、查全率等指标。绘制MLP模型在训练集和验证集组合训练过程中的损失函数Loss曲线。

from sklearn.neural_network import MLPClassifier
from sklearn.metrics import classification_report

# 训练集、验证集特征及标签组合
X_train_val_feats = None
y_train_val = None

……