libsvm 多分类中model参数

一、先看REDEME中的介绍 

struct svm_model

{

struct svm_parameter param; /* parameter */
int nr_class; /* number of classes, = 2 in regression/one class svm */
int l; /* total #SV */
struct svm_node **SV; /* SVs (SV[l]) */
double **sv_coef; /* coefficients for SVs in decision functions (sv_coef[k-1][l]) */
double *rho; /* constants in decision functions (rho[k*(k-1)/2]) */
double *probA; /* pairwise probability information */
double *probB;
int *sv_indices;        /* sv_indices[0,...,nSV-1] are values in [1,...,num_traning_data] to indicate SVs in the training set */

/* for classification only */

int *label; /* label of each class (label[k]) */
int *nSV; /* number of SVs for each class (nSV[k]) */
/* nSV[0] + nSV[1] + ... + nSV[k-1] = l */
/* XXX */
int free_sv; /* 1 if svm_model is created by svm_load_model*/
/* 0 if svm_model is created by svm_train */
};
    param describes the parameters used to obtain the model.

    nr_class is the number of classes. It is 2 for regression and one-class SVM.

    l is the number of support vectors. SV and sv_coef are support
    vectors and the corresponding coefficients, respectively. Assume there are
    k classes. For data in class j, the corresponding sv_coef includes (k-1) y*alpha vectors,
    where alpha's are solutions of the following two class problems:
    1 vs j, 2 vs j, ..., j-1 vs j, j vs j+1, j vs j+2, ..., j vs k
    and y=1 for the first j-1 vectors, while y=-1 for the remaining k-j 
    vectors. For example, if there are 4 classes, sv_coef and SV are like:

        +-+-+-+--------------------+
        |1|1|1|                    |
        |v|v|v|  SVs from class 1  |
        |2|3|4|                    |
        +-+-+-+--------------------+
        |1|2|2|                    |
        |v|v|v|  SVs from class 2  |
        |2|3|4|                    |
        +-+-+-+--------------------+
        |1|2|3|                    |
        |v|v|v|  SVs from class 3  |
        |3|3|4|                    |
        +-+-+-+--------------------+
        |1|2|3|                    |
        |v|v|v|  SVs from class 4  |
        |4|4|4|                    |
        +-+-+-+--------------------+
    rho is the bias term (-b). probA and probB are parameters used in
    probability outputs. If there are k classes, there are k*(k-1)/2
    binary problems as well as rho, probA, and probB values. They are
    aligned in the order of binary problems:
    1 vs 2, 1 vs 3, ..., 1 vs k, 2 vs 3, ..., 2 vs k, ..., k-1 vs k.
    sv_indices[0,...,nSV-1] are values in [1,...,num_traning_data] to
    indicate support vectors in the training set.
    label contains labels in the training data.
    nSV is the number of support vectors in each class.

二、基本概念

  什么是支持向量？ 存在y = +1 的支持向量  y=-1的支持向量 

  svm对多分类支持？用1vs1加上投票的方法来确定

三、二分类和多分类model参数 

对于二分类

SV用于保持支持向量
sv_coef用于保存支持向量对应的系数yi*alpha

因为C-SVC 的决策函数如下：可以看到wi是标量，Xi是 支持向量 

 

因此 SV 和 SV_coef的行数是相同的，假设为M，则SV 为M *（fetures个数） SV_coef 为 M * 1，注意这里的支持向量分正支持向量和 负支持向量，由nSV指出个数，这个概念在推广到多分类是很有用。

       +-+-+-+--------------------+
        |1|                |
        |v|  SVs from class 1  |
        |2|                 |
        +-+-+-+--------------------+

对于多分类

SV用于保持支持向量
sv_coef用于保存支持向量对应的系数yi*alpha

因为C-SVC 的决策函数如下：可以看到wi是标量，Xi是 支持向量 

       k -1            fetures 个数  

        +-+-+-+         -------------------+
        |1|1|1|                    |
        |v|v|v|           SVs from class 1  |           假设这里有  nSV[0]个支持向量 
        |2|3|4|                    |

        | | | | | |       
        +-+-+ |+       --------------------+
        |1|2|2|                    |
        |v|v|v|  S      Vs from class 2  |              nSV[1]个支持向量
        |2|3|4|                    |
        +-+-+-+--      ------------------+
        |1|2|3|                    |
        |v|v|v|           SVs from class 3  |            nSV[2]个支持向量
        |3|3|4|                    |

        | | | | | | | |        | | | | | | | | | | | | | | | |     

        | | | | | | | |       |  | | | | | | | | | | | | | 

        +-+-+-+---       -----------------+
        |1|2|3|                    |
        |v|v|v|            SVs from class 4  |           nSV[3]个支持向量
        |4|4|4|                    |
        +-+-+-+----       ----------------+

For data in class j, the corresponding sv_coef includes (k-1) y*alpha vectors,
    where alpha's are solutions of the following two class problems:
    1 vs j, 2 vs j, ..., j-1 vs j, j vs j+1, j vs j+2, ..., j vs k
    and y=1 for the first j-1 vectors, while y=-1 for the remaining k-j 
    vectors.

因此 SV 和 SV_coef的行数是相同的，注意由于使用1VS1 策略，需要分别计算 

1 vs j, 2 vs j, ..., j-1 vs j, j vs j+1, j vs j+2, ..., j vs k 这么多分类器的支持向量和系数。

由于有k个分类器，因此对每个类需要（k-1）个 （y*alpha vectors）。

在这个k-1个系数向量中，有j-1个 是i vs j  and i < j 的 ，此时，i中有正支持向量，有 k-j 个是 i vs j  and i>j 的，此时i中有负支持向量。因此假如要计算 i vs j 分类器的w 系数，需要  class i中的 i vs j 系数向量，class i中的 支持向量 ，也需要 class j中的 j vs i系数向量，class j中的支持向量 ，注意这两个向量是互补的，一个在分类线左边，一个在右边，也就是上面说的正支持向量和负的支持向量。只要组合这些向量就能恢复出二分类中的中的系数向量和支持向量。

Q：libsvm 文档摘要

Q:How could i generate the primal variable w oflinear svm?

Let's start from the binary class and assume you have two labels -1 and +1. After obtaining the model from calling svmtrain, do the following to have w and b:

w = model.SVs' * model.sv_coef;
b = -model.rho;

if model.Label(1) == -1
  w = -w;
  b = -b;
end

If you do regression or one-class SVM, then the if statement is not needed.

For multi-class SVM, we illustrate the setting in the following example of running the iris data, which have 3 classes

  
> [y, x] = libsvmread('../../htdocs/libsvmtools/datasets/multiclass/iris.scale');
> m = svmtrain(y, x, '-t 0')

m = 

    Parameters: [5x1 double]
      nr_class: 3
       totalSV: 42
           rho: [3x1 double]
         Label: [3x1 double]
         ProbA: []
         ProbB: []
           nSV: [3x1 double]
       sv_coef: [42x2 double]
           SVs: [42x4 double]

sv_coef is like:

+-+-+--------------------+
|1|1|                    |
|v|v|  SVs from class 1  |
|2|3|                    |
+-+-+--------------------+
|1|2|                    |
|v|v|  SVs from class 2  |
|2|3|                    |
+-+-+--------------------+
|1|2|                    |
|v|v|  SVs from class 3  |
|3|3|                    |
+-+-+--------------------+

so we need to see nSV of each classes.

  
> m.nSV

ans =

     3
    21
    18

Suppose the goal is to find the vector w of classes 1 vs 3. Then y_i alpha_i of training 1 vs 3 are

  
> coef = [m.sv_coef(1:3,2); m.sv_coef(25:42,1)];

and SVs are:

  
> SVs = [m.SVs(1:3,:); m.SVs(25:42,:)];

Hence, w is

> w = SVs'*coef;

For rho,

> m.rho

ans =

    1.1465
    0.3682
   -1.9969
> b = -m.rho(2);

because rho is arranged by 1vs2 1vs3 2vs3.