Computer vision 1

1.SVM

1. Lagrangian Theory
   Given an optimization problem with the objective function f(w) and equality constrains $g_i$(w)=0

$$\begin{gathered} L(w,\alpha )=f(w)-\sum _{i=1}^n{a}_i{g}_i(w) \\ L(w,b,a)=\frac{1}{2}\parallel w{\parallel }^2-\stackrel{N}{\underset{i=1}{\sum a_i(y_i(w\cdot x_i+b)-1)}} \end{gathered}$$

2. Karush-Kuhn-tacker(KKT) constraint
   For a solution in nonlinear programming, consider the following nonlinear minimization or maximization problem

$$\begin{array}{l}\frac{\partial L(w,\alpha )}{\partial w}=0\\ g_i(w^{\ast })\ge 0\\ {\alpha }_i\ge 0\\ {\alpha }_i{g}_i(w^{\ast })=0\end{array}$$

3. Support Vector

So if the points not on the boundary do not contribute and the${\alpha }_i$will be 0

$$L(w,b,\alpha )=\sum _{i=1}^n{\alpha }_i-\frac{1}{2}\sum _{i,j=1}^n{\alpha }_i{\alpha }_j{x}_i{x}_j{y}_i{y}_j$$

4. Kernel Trick

• To solve the problem calculate in high-dimension: Project data to much higher dimension space so that a hyper-plane can be found/eliminate the computationally expensive operations in the higher dimension
• For example: Polynomial equivalent to project to six-dimensional space

2 Random forest

1. Bagging - Bootstrap Aggregation

• The training dataset is a difference while using the same algorithms
• The random forest takes advantage of this by allowing each individual tree to randomly sample from the dataset with replacement, resulting in different trees. This process is known as bagging.

2. Random Forest

• The training dataset is the same while the algorithms are different
• Each tree in a random forest can pick only from a random subset of features. This forces even more variation amongst the trees in the model and ultimately results in lower correlation across trees and more diversification.

3. the basic theory of random forest

• The reason for this wonderful effect is that the tree protect each other from their individual error
• Two prerequisites: actual signal in feature and prediction made by the individual have low correlation with each other

3 Tips

• Conduct validation
• Conduct cross-validation : the training is small
• Conduct leave-one-out: Training with all the data except one and repeat the process