f
θ
:
x
→
y
f_{\theta}: x\rightarrow y
fθ:x→y
y
y
y通常可以分为两类:离散或连续
连续时为回归问题
linear regression
loss function
gradient descent
loss surface
convex Optimization
- linear regression
- linear Regression
- logistic Regression
- Classification
回归问题实战
-
线性回归
-
离散值预测
-
Image Classificaton
- ImageNet
- MNIST
- Hand-written Digits Recognition
- 7000 images per category
- train/test:60k/10k
-
Input and Output
- flatten
- matrix to vector
- one-hot
- ∑ p ( y = i ∣ x ) = 1 \sum{p(y = i | x)} = 1 ∑p(y=i∣x)=1
- flatten
-
-
Regression vs Classification
- y ∈ R d y \in R^d y∈Rd
- y [ i ] = p ( y = i ∣ x ) y[i] = p(y = i| x) y[i]=p(y=i∣x)
-
unlinear
- ReLU
-
hidden layer stack
-
Loss
- MSE(out, label)
- Cross Entropy
-
GPU speedup!
-
Enjoy MNIST!