Four branches of machine learning
Supervised learning
- optical character recognition
- speech recognition
- image classification
- language translation
- Sequence generate
Given a picture, predict a caption describing it.
- Syntax tree prediction
Given a sentence, predict its decomposition into a syntax tree.
- Object detection
- Image segmentation
Given a picture, draw a pixel-level mask on a specific object.
Unsupervised learning
- dimensionality reduction.
- clustering
Self-supervised learning
Supervised learning without human-annotated labels.
- autoencoder
Reinforcement learning
An agent receives information about its environment and learns to choose actions that will maximize some reward.
The universal workflow of machine learning
- Defining the problem and assembling a dataset.
- Choosing a measure of success.
- Deciding on an evaluation protocol.
- Preparing your data.
- Developing a model that does better than a baseline.
- Scaling up: developing a model that overfits.
- Regularizing your model and tuning your hyperparameters.
Overfitting
Reducing the network’s size
The general workflow to find an appropriate model size is to start with relatively few layers and parameters, and increase the size of the layers or add new layers until you see diminishing returns with regard to validation loss.
Adding weight regularization
Put constrains on the complexity of a network by forcing its weights to take only small values, which makes the distribution of weight values more regular.
- L1 regularization——The cost added is proportional to the absolute value of the weight coefficients.
- L2 regularization——******the square of the value of the weight coefficients.
Adding dropout
Dropout, applied to a layer, consists of randomly dropping out (setting to zero) a number of output features of the layer during training.