1、使用svm对二维数据集进行分类
①使用线性边界函数进行分类
训练数据的可视化函数:
function plotData(X, y)
%PLOTDATA Plots the data points X and y into a new figure
% PLOTDATA(x,y) plots the data points with + for the positive examples
% and o for the negative examples. X is assumed to be a Mx2 matrix.
%
% Note: This was slightly modified such that it expects y = 1 or y = 0
% Find Indices of Positive and Negative Examples
pos = find(y == 1); neg = find(y == 0);
% Plot Examples
plot(X(pos, 1), X(pos, 2), 'k+','LineWidth', 1, 'MarkerSize', 7)
hold on;
plot(X(neg, 1), X(neg, 2), 'ko', 'MarkerFaceColor', 'y', 'MarkerSize', 7)
hold off;
end
在命令行运行代码对于第一组数据进行数据可视化
load('ex6data1.mat');
plotData(X,y);
效果如图所示:
训练svm分类器
当C=1时,在命令行执行语句进行训练分类器并进行边界可视化:
model=svmTrain(X,y,C,@linearKernel,0.001,20);
visualizeBoundaryLinear(X,y,model);
由于C比较小,欠拟合,有一个点并没有进行正确的分类,效果如图:
当C=100时,效果如图:
②使用非线性分界函数进行分类
训练数据可视化如图:
对于线性不可分的使用高斯核函数进行训练
单独使用训练数据中的特征变量不能进行区分,所以需要增加一些新的特征变量,可以使用这些特征的高阶多项式进行划分,但是运算量非常大。
可以通过标记点以及核函数定义新的特征变量,从而训练复杂的非线性边界,在本篇使用高斯内核函数。
高斯内核函数的实现:
function sim = gaussianKernel(x1, x2, sigma)
%RBFKERNEL returns a radial basis function kernel between x1 and x2
% sim = gaussianKernel(x1, x2) returns a gaussian kernel between x1 and x2
% and returns the value in sim
% Ensure that x1 and x2 are column vectors
x1 = x1(:); x2 = x2(:);
% You need to return the following variables correctly.
sim = 0;
% ====================== YOUR CODE HERE ======================
% Instructions: Fill in this function to return the similarity between x1
% and x2 computed using a Gaussian kernel with bandwidth
% sigma
%
%
sim=exp(-1.0*(x1-x2)'*(x1-x2)/(2.0*sigma*sigma));
% =============================================================
end
使用高斯核函数进行训练非线性分类器
C=1;
sigma=0.1;
svmTrain(X,y,C,@(x1,x2)gaussianKernel(x1,x2,sigma));
visualizeBoundary(X,y,model);
3、使用交叉验证选择合适的C和δ值。
使用X,y做为训练集,Xval,yval做为测试集来选择合适的C和δ值
function [C, sigma] = dataset3Params(X, y, Xval, yval)
%DATASET3PARAMS returns your choice of C and sigma for Part 3 of the exercise
%where you select the optimal (C, sigma) learning parameters to use for SVM
%with RBF kernel
% [C, sigma] = DATASET3PARAMS(X, y, Xval, yval) returns your choice of C and
% sigma. You should complete this function to return the optimal C and
% sigma based on a cross-validation set.
%
% You need to return the following variables correctly.
C = 1;
sigma = 0.3;
% ====================== YOUR CODE HERE ======================
% Instructions: Fill in this function to return the optimal C and sigma
% learning parameters found using the cross validation set.
% You can use svmPredict to predict the labels on the cross
% validation set. For example,
% predictions = svmPredict(model, Xval);
% will return the predictions on the cross validation set.
%
% Note: You can compute the prediction error using
% mean(double(predictions ~= yval))
%
wrong=1;
eg=[0.01,0.03,0.1,0.3,1,3,10,30];
for i=1:length(eg)
for j=1:length(eg)
model=svmTrain(X,y,eg(i),@(x1,x2)gaussianKernel(x1,x2,eg(j)));
predictions=svmPredict(model,Xval);
twrong=mean(double(predictions~=yval));
if twrong<wrong
wrong=twrong;
C=eg(i);
sigma=eg(j);
end
end
end
% =========================================================================
end
使用得到的C和δ值进行svm分类器的训练,并进行边界划分。
[C, sigma] = dataset3Params(X, y, Xval, yval);
model = svmTrain(X, y, C, @(x1, x2)gaussianKernel(x1, x2, sigma));
visualizeBoundary(X, y, model);