SVM

“Support Vector Machine” (SVM) is a supervised machine learning algorithm which can be used for both classification or regression challenges. However,  it is mostly used in classification problems. It will find out the hyper-plane that differentiate the two classes very well.

the sample vectors that can satisfy the below formulas are called "support vector"

H1：y(wTx+b)=+1and  H2：y(wTx+b)=−1

In one word, we will try to find out a situation that should maximize the minimum margin between support vectors and hyper plane  The final question gonna be transformed like this:

How to compute the above formula? Here are lots of complex mathematics process behind it, see the reference below:   

http://blog.csdn.net/SJ1820161738/article/details/76038711?locationNum=1&fps=1

When the classification question does not have a linear hyper plane in current dimension, we will use The Kernel Trick.

The Kernel trick involves kernel functions that can be enable in higher-dimension spaces without explicitly calculating the coordinates of points within that dimension: instead, kernel functions compute the inner products between the images of all pairs of data in a feature space. this allows them the very useful attribute of calculating the coordinates of higher dimensions, In other word dramatically reduce the time complexity of the algorithm. Using the kernel trick enables us  effectivelyrun algorithm in a high-dimensional space with lower-dimensional data.

Pros and Cons associated with SVM

• Pros:
  • It works really well with clear margin of separation
  • It is effective in high dimensional spaces.
  • It is effective in cases where number of dimensions is greater than the number of samples.
  • It uses a subset of training points in the decision function (called support vectors), so it is also memory efficient.
• Cons:
  • It doesn’t perform well, when we have large data set because the required training time is higher
  • It also doesn’t perform very well, when the data set has more noise i.e. target classes are overlapping
  • SVM doesn’t directly provide probability estimates, these are calculated using an expensive five-fold cross-validation. It is related SVC method of Python scikit-learn library.

SMO(Sequential minimal Optimization) is an algorithm for solving the quadratic programming(QP) problem that arises during the training of support vector machines.