2022/4/24第三次周报

1.机器学习任务攻略

General Guidance 一般指导
检查training data 的 loss
1.1large
1.1.1 模型偏差 model bias
The model is too simple
Solution:redesign your model to make it more flexible
模型太简单
解决方案：重新设计模型，使其更加灵活

1.1.2优化不好 optimization lssue
Solution:more powerful optimization technology
解决方案：更强大的优化技术
1.2 small
检查loss on testing data
1.2.1large
Overfitting
Solution:
增加训练资料more training data
(Data augmentation 数据增强)
给model一些限制
constrainde model:
less parameters,sharing parameters
less features
early stopping
regularization
dropout
mismatch(大多数情况不会出现)
your training and testing data have different distributions
您的训练和测试 data 有不同的分布
1.1.2 small

2.Optimization Fails

判断一个点是local minima 还是 saddle point

Hessian
判断critical point 是 local minima 还是 saddle point

critical point ：gradient is close to zero

如果 critical point 是 saddle point 的话，不用担心

3.Batch 和 Momentum 训练技巧

3.1Batch

small batch vs big batch

3.2Momentum