周志华Watermelon Book SVM部分公式补充、一些原理解释

Z.F Zhou Watermelon Book SVM.This article is a supplementary material for SVM.

Watermelon Book

6.3

Suppose training data set is linear separable.Minimum margin is $\delta$.
$$\begin{gathered} \{\begin{array}{l}{w}^{T}x+b>=+\delta ,y_i=+1\\ w^{T}x+b<=-\delta ,y_i=-1\end{array} \\ \to \{\begin{array}{l}\frac{w^T}{\delta }x+\frac{b}{\delta }>=+1,y_i=+1\\ \frac{w^T}{\delta }x+\frac{b}{\delta }<=-1,y_i=-1\end{array} \end{gathered}$$

6.8

Refer to the below link about lagrange and KKT.

6.9

$$\begin{gathered} L=1/2w^{T}w+\sum _{i=1}^m{a}_i(1-y_i(w^t{x}_i+b)) \\ derivation \\ 0=w-\sum _{i=1}^m{a}_i{y}_i{x}_i \\ \to w=\sum _{i=1}^m{a}_i{y}_i{x}_i \end{gathered}$$

6.11

You can compute $\frac{1}{2}||w|{|}^2$ and ${\sum }_{i=1}^m{a}_i(1-y_i(w^t{x}_i+b))$ respectively.I am lazy.

6.18

Because the $a_i$ may have error with theory value using SMO.

6.41

In below of 6.41,the interpretation about support vectors is based on $a_i!=0$.

[1] Partial Reference

KKT

part1

What is clear interpretation is:
Three equations base on:
$u=0org=0whereuandgarevectors.(u_i=0or{g}_i=0)$
For three equations,we can compute a $x$ minimizing $f(x)$ by compute another formulation,e,g,${\min }_x{\max }_{u}L(x,u)$.Their $x$ and $u$ are common.

part2

Why is the $L(\hat{x},u)$ equivalent to the ${\min }_{x}L(x,u)$?
I think it can use proof by contradiction.Suppose $L(x^{\prime },u)={\min }_{x}L(x,u)$.Then ${\max }_u{\max }_{x}L(x,u)={\max }_{u}L(x^{\prime },u)=f(x^{\prime })!=f(\hat{x})$.It is contradictory.

Lagrange

I think the link in below is very detailed.

Reference

[1]Lagrange and KKT condition.
[2]拉格朗日乘数