Deep Learning Workflow吴恩达深度学习课程笔记

Step 0: Data division(training, dev(target), test, train-dev set)- Data mismatch
- artificial data synthesis
Single real number evaluation metric( optimizing and satisficing metrics)
Approaches to update parameters(panda, caviar)

Step 1: Model selection( hyperparameter tuning included) and parameter initialization
-Softmax regression
-Transfer learning(small or lots of data)
Note: find an architecture from open source and transfer learning. Freeze previous layers(depend on data size) and save to disk
-multitask learning
-CNN—convolution
–pooling(reduce size of representation to speed up computation)
–fully connected
Parameter initialization: zeros, random, He, Xavier

Step 2: Optimization algorithm (mini-batch, momentum, RMSprop, Adam)( hyperparameter tuning included)

Step 3: Speed up training- input normalization, input vectorization, learning rate decay(slowly reduce learning rate over time, not preferable), a different optimization algorithm, batch normalization
Gradient checking(once after several iterations during training, slow, once prove correctness, turn off)- avoid gradient exploding or vanishing(random weight initialization, intermediate normalization, skip connections(ResNets) for DNN ,auxiliary classifiers)
Keep iterating to improve dev set performance and run gradient descent longer
Ensure convergence

Step 4: (training data) High bias -train it longer
-bigger network
-(NN architecture search not recommended due to performance hard to predict)/hyperparameters search
(training data) high variance - more training data
-regularization(L1,L2(lambda bigger might cause oversmooth and bias) ,dropout(once every iteration& only in training) , maxout, data augmentation, early stopping)
-(NN architecture search not recommended)/hyperparameters search
Test set not fit well(over-fit of dev set) - bigger dev set
Not perform well in real world - change either the dev set or the cost function

Step 5: Manual error analysis to prioritize
- decide what’s most important promising direction

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值