机器学习笔记 ---- Support Vector Machines

Support Vector Machines

1. Cost Function

$$J(\theta )=C\left[\sum _{i=1}^m{y}^{(i)}{\text{cost}}_1({\theta }^T{x}^{(i)})+(1-y^{(i)}){\text{cost}}_0({\theta }^T{x}^{(i)})\right]+\frac{1}{2}\sum _{j=1}^n{\theta }_j^2$$

2. Hypothesis

$$f(x)=\{\begin{array}{ll}1& \text{if θTx>=0}\\ 0& \text{otherwise}\end{array}$$

3. Margin of SVM

4. Kernels

Define landmarks $l$



Using $f$ and $\theta$ when making predictions…

5. How to Get Landmarks

One way is to use the first m training examples.

6. The Effects of Parameters in SVM

1) For $C$
Large C : λ small, low bias, high variance
Small C : λ big, high bias, low variance
2) For ${\sigma }^2$
Large ${\sigma }^2$ : high bias, low variance
Small ${\sigma }^2$ : low bias, high variance

7. Choice of Kernal

Need to satisfy Mercer’s Theorem.

1. No kernal (Linear Kernal)
   when n is large/ n is small && m is large
2. Gaussian Kernal
   when n is small, m is intermediate
   Need to use feature scaling before using!
3. Other Alternative Choices:
   Polynomial Kernal, String Kernal, Chi-Square Kernal, Intersection Kernal…