1 Practical aspects of Deep Learning
1.1 Train/dev/test set
- train set
- hold-out cross validation set/development set/dev set
- test set
In the previous era of machine learning, 60/20/20% is a good split for train/dev/test set. But in the modern big data era, 95/2.5/2.5% even 98/1/1% is better.
1.2 Bias and Variance
trade-off of bias and variancd
Train set error | 1% | 15% | 15% | 0.5% |
---|---|---|---|---|
Dev set error | 1% | 16% | 30% | 1% |
bias&variance | high variance | high bias | both high | both low |
1.3 Basic “recipe” for machine learning
If high bias, you can:
- try pick a network such as more hidden layers or more hidden units or you could train it longer.
- finde a new network architecture.
If high variance, you can:
- get more data
- regularization
- find a more appropriate neural network