机器学习笔记之Kernelized Support Vectors Machines

最新推荐文章于 2023-02-08 16:15:37 发布

YukAgame

最新推荐文章于 2023-02-08 16:15:37 发布

阅读量242

点赞数

分类专栏：机器学习学习笔记

本文链接：https://blog.csdn.net/weixin_38686737/article/details/108082245

版权

机器学习学习笔记专栏收录该内容

7 篇文章 2 订阅

订阅专栏

机器学习笔记之Kernelized Support Vectors Machines

(这里书上说的太简略了，三脸懵逼）

Kernelized support vector machines are an extension that allows more complex models that are not simply defined by hyper planes in the space
Adding nonlinear features to representation of the data can make
linear models more powerful.
Mainly two ways to map the data into higher-dimensional space

Polynomial kernel: compute all possible polynomials
Radial basis function(RBF) model: considers polynomials but importance of features decreases for higher degrees
Only a subset of data points to define the decision boundary: the ones that lie on the border of between classed. (called support vectors

在Python当中可以通过scikit-learn中的 SVC来实现

from sklearn.svm import SVC
import mglearn
import matplotlib.pyplot as plt
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split

x, y = mglearn.tools.make_handcrafted_dataset()
'''gamma是指GAUSSIAN KERNEL下的RADIUS,c指模型的复杂性'''
svm = SVC(kernel = 'rbf', C = 10, gamma = 0.1).fit(x,y)

mglearn.plots.plot_2d_separator(svm,x,eps = .5)
mglearn.discrete_scatter(x[:,0],x[:,1],y)
sv = svm.support_vectors_
sv_labels = svm.dual_coef_.ravel() >0
mglearn.discrete_scatter(sv[:,0],sv[:,1],sv_labels,s = 15, markeredgewidth=3)
plt.xlabel('Feature0')
plt.ylabel('Feature1')
plt.show()

下图是默认C=10,GAMMA=0.1时的分类情况

在这里插入图片描述

通过变化C和gamma的值来看看具体影响

fig,axes = plt.subplots(3,3,figsize = (15,10))

for ax,C in zip(axes,[-1,0,3]):
    for a ,gamma in zip(ax,range(-1,2)):
        mglearn.plots.plot_svm(log_C=C,log_gamma=gamma, ax = a)
axes[0,0].legend(['Class0,Class1','sv class0','sv class1'],ncol =4, loc = (-9,1.2))
plt.show()

可以看到随着C和GAMMA的增大，分界线变得越来越曲线化

在这里插入图片描述

还是以乳腺癌人群的数据为例，

cancer = load_breast_cancer()
x_train,x_test,y_train,y_test = train_test_split(cancer.data,cancer.target,random_state = 0)
sv = SVC().fit(x_train,y_train)
plt.boxplot(x_train)
plt.yscale('symlog')
plt.xlabel('Feature Index')
plt.ylabel('Feature Magnitude')
plt.show()

可以看出这个数据集的magnitude差距很大，这给kernel SVM带来了很差的影响（测试集表现只有0.67），这意味着需要进行一些数据的前处理

在这里插入图片描述

min_on_training = x_train.min(axis =0)
range_on_training = (x_train - min_on_training).max(axis = 0)
x_train_scaled = (x_train - min_on_training)/range_on_training
x_test_scaled = (x_test - min_on_training)/range_on_training
svc =SVC()
svc.fit(x_train_scaled,y_train)
print('Training score:%s'%(svc.score(x_train_scaled,y_train)),'Test score:%s'%(svc.score(x_test_scaled,y_test)))