VLFeat教程SVM

VLFeat includes fast SVM solvers, SGC [1] and (S)DCA [2], both implemented in vl_svmtrain. The function also implements features, like Homogeneous kernel map expansion and SVM online statistics. (S)DCA can also be used with different loss functions.

Support vector machine

A simple example on how to use vl_svmtrain is presented below. Let's first load and plot the training data:

% Load training data X and their labels y
vl_setup demo % to load the demo data
load('vl_demo_svm_data.mat');

Xp = X(:,y==1);
Xn = X(:,y==-1);

figure
plot(Xn(1,:),Xn(2,:),'*r')
hold on
plot(Xp(1,:),Xp(2,:),'*b')
axis equal ;

Now we have a plot of the tutorial training data:

Training Data.

Now we will set the learning parameters:

lambda = 0.01 ; % Regularization parameter
maxIter = 1000 ; % Maximum number of iterations

Learning a linear classifier can be easily done with the following 1 line of code:

[w b info] = vl_svmtrain(X, y, lambda, 'MaxNumIterations', maxIter)

Now we can plot the output model over the training data.

% Visualisation
eq = [num2str(w(1)) '*x+' num2str(w(2)) '*y+' num2str(b)];
line = ezplot(eq, [-0.9 0.9 -0.9 0.9]);
set(line, 'Color', [0 0.8 0],'linewidth', 2);

The result is plotted in the following figure.

Learned model.

The output info is a struct containing some statistic on the learned SVM:

info =

            solver: 'sdca'
            lambda: 0.0100
    biasMultiplier: 1
              bias: 0.0657
         objective: 0.2105
       regularizer: 0.0726
              loss: 0.1379
     dualObjective: 0.2016
          dualLoss: 0.2742
        dualityGap: 0.0088
         iteration: 525
             epoch: 3
       elapsedTime: 0.0300

It is also possible to use under some assumptions [3] a homogeneous kernel map expanded online inside the solver. This can be done with the following commands:

% create a structure with kernel map parameters
hom.kernel = 'KChi2';
hom.order = 2;
% create the dataset structure
dataset = vl_svmdataset(X, 'homkermap', hom);
% learn the SVM with online kernel map expansion using the dataset structure
[w b info] = vl_svmtrain(dataset, y, lambda, 'MaxNumIterations', maxIter)

The above code creates a training set without applying any homogeneous kernel map to the data. When the solver is called it will expand each data point with a Chi Squared kernel of period 2.

Diagnostics

VLFeat allows to get statistics during the training process. It is sufficient to pass a function handle to the solver. The function will be then called every DiagnosticFrequency time.

(S)DCA diagnostics also provides the duality gap value (the difference between primal and dual energy), which is the upper bound of the primal task sub-optimality.

% Diagnostic function
function diagnostics(svm)
  energy = [energy [svm.objective ; svm.dualObjective ; svm.dualityGap ] ] ;
end

% Training the SVM
energy = [] ;
[w b info] = vl_svmtrain(X, y, lambda,...
                           'MaxNumIterations',maxIter,...
                           'DiagnosticFunction',@diagnostics,...
                           'DiagnosticFrequency',1)

The objective values for the past iterations are kept in the matrix energy. Now we can plot the objective values from the learning process.

figure
hold on
plot(energy(1,:),'--b') ;
plot(energy(2,:),'-.g') ;
plot(energy(3,:),'r') ;
legend('Primal objective','Dual objective','Duality gap')
xlabel('Diagnostics iteration')
ylabel('Energy')
SVM objective values plot.

References

  • [1] Y. Singer and N. Srebro. Pegasos: Primal estimated sub-gradient solver for SVM. In Proc. ICML, 2007.
  • [2] S. Shalev-Schwartz and T. Zhang. Stochastic Dual Coordinate Ascent Methods for Regularized Loss Minimization. 2013.
  • [3] A. Vedaldi and A. Zisserman. Efficient additive kernels via explicit feature maps. In PAMI, 2011.
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值