KKB_python_scratch_framework_sumup_lesson1

M IT Code is for people to review, and runs sometimes.

Review

1. Corralation, salient feature

COV(X,Y) = E(X-E(X)*(Y-E(Y)))
r(X,Y) = COV(X,Y)/(STD(X)*STD(Y)) == CORR

2. Machine Learning

2.1 Computational intelligence

2.2 Perceptual intelligence

2.3 Cognitive intelligence

3. K-Neighbor-Nearest(Lazy learning)

items = Sort(square(input - X)) 
yhat = mean(items(top_k))

4 function Fitting

4.1 Calculus

4.2 Random Iteration Get Best K and B(MonteCarlo Simulation)

  min_loss = +inf
  do 
      loss_ = MSE(Y, X * random_k + random_b)
      if loss_ < min_loss:
          min_loss = loss_
          best_k = random_k
          best_b = random_b
  while(1)        

5. Loss

5.1 Mean Square Error(MSE)

Partial of MSE:

 dL/dk = 2 * mean((yi - xi * k + b) * (-xi))
 dL/db = 2 * mean((yi - xi * k + b) * (1))

6. Gradient Descent

 k = k + (-1) * dL/dk
 b = b + (-1) * dL/db

7. Performance of Gradient Descent

 yhat1 = k * x + b ( < 1ns)
 yhat2 = knn(x) (~= 1ms)
 yhat2 is 1e6 times of yhat1