Machine Learning（李宏毅公开课笔记）-Machine Learning and Deep Learning

Machine Learning and Deep Learning

1. Functions

1. Regression：PM2.5
2. Classification：chess
3. Others：structured learning

2. The procedures of finding the functions

1. Functions with unknown parameters
2. Define loss from training data
3. Optimization

   1. gradient descent
      A) randomly set an initial value w
      B) compute the $$\partial l/\partial w$$
      C) update w iteratively

3. Models

1. linear model
2. sophisticated model

1. linear curves:
   all piecewise linear curves = constant + sum of set (sigmoid)
   activation function:
   1.hard sigmoid: which can be represented by sum of two ReLU
   
   
   2.rectified linear unit(ReLU):$$max(0,wx+b)$$
   
   3.soft sigmoid: $$\frac{c}{1+e^{-(wx+b)}}=c\ast sigmoid(wx+b)$$
   

2. Beyond piecewise curves
   
   approximate continuous curve by a piecewise linear curve
   to have a good approximate, we need sufficient pieces

3. New model: More Features
   $$y=b+\sum _i{c}_i\ast sigmoid(\sum _j{w}_{ij}{x}_j+b_i)$$
   $$r_i=W_{i}X+b_i，a_i=sigmoid(ri)$$
   $$y=b+CA$$
   optimization of new model:
   $$\Theta =[WBC]$$
   $$gradient=\left|\begin{array}{c}\frac{\partial L}{\partial {\Theta }_1}\\ \frac{\partial L}{\partial {\Theta }_2}\\ ...\\ \frac{\partial L}{\partial {\Theta }_n}\end{array}\right|$$
   $$g=\nabla L({\Theta }^0)$$
   $$\left|\begin{array}{c}{\Theta }_1^1\\ {\Theta }_2^1\\ ...\\ {\Theta }_n^1\end{array}\right|=\left|\begin{array}{c}{\Theta }_1^0\\ {\Theta }_2^0\\ ...\\ {\Theta }_n^0\end{array}\right|-\eta \ast g$$

4. epoch | batch | update | iteration
   number of samples: 1000
   batch size: 10
   iterations: 100
   epoch:1

video link: https://speech.ee.ntu.edu.tw/~hylee/ml/2021-spring.html.