DL-2-week1-Practical aspects of Deep Learning

1 Practical aspects of Deep Learning

1.1 Train/dev/test set

• train set
• hold-out cross validation set/development set/dev set
• test set
  In the previous era of machine learning, 60/20/20% is a good split for train/dev/test set. But in the modern big data era, 95/2.5/2.5% even 98/1/1% is better.

1.2 Bias and Variance

trade-off of bias and variancd

Train set error	1%	15%	15%	0.5%
Dev set error	1%	16%	30%	1%
bias&variance	high variance	high bias	both high	both low

1.3 Basic “recipe” for machine learning

If high bias, you can:
- try pick a network such as more hidden layers or more hidden units or you could train it longer.
- finde a new network architecture.
If high variance, you can:
- get more data
- regularization
- find a more appropriate neural network

1.4 Regularization