machine learning ≈ looking for function
Different types of functions:
Regression(回归): The function outputs a scalar
Classification(分类): Given options(classes),the function outputs the correct one
structured learning(产生结构,创造)
How to find a function?
1.Function with unknown parameters
Model :
y是要预测的,x是已知的,b和w是要去寻找的(learn from data)
2.Define Loss from Training Data
Loss is a function of parameters ( L(b,w) )
Loss: how good a set of value is.
表示每组数据预测跟真实数据之间的误差 |
-
|
Loss:L =
Label 是正确的数值
3.Optimization(最优解)
找到![{w}^*,{b}^* = argminL](https://latex.csdn.net/eq?%5Cdpi%7B150%7D%20%7Bw%7D%5E*%2C%7Bb%7D%5E*%20%3D%20argminL)
Gradient descent(梯度下降法)
1.(Randomly)Pick an inital value
2.Compute
Negative -- increase w
Positive -- decrease w
(步长)
: learning rate
整个过程中自己设定的东西叫hyperparameter
3. Update w iteratively
Linear models have severe limitation : Model Bias
激活函数
Sigmoid Function: y = csigmoid(b+w)
ReLU
New Model : More Features
1.Function with unknown
2.Define loss from training data
Loss :
3.Optimization
1.(Randomly)Pick initial values
2.Compute gradient g
(gradient)
简写为
Batch : 将一组数据分成很多个batch,它的size自己决定
这样可以得到L1, 根据L1计算 .....
1 epotch = see all the batches once
Rectified Linear Unit(ReLU) : cmax(0,b + wx1)
Sigmoid ReLU
只要够多的激活函数就可以逼近任何function