Where Transfer Learning from A to B Makes Sense
- Task A and B have the same input X.
- You have a lot more data for A than B.
- Low level features in A could be helpful for learning B.
Where Multi-task Learning Makes Sense
- Training on a set of tasks that could benefit from having shared lower-level features.
- Usually: Amount of data you have for each task is quite similar.
- Can train a big enough neural network to do well on all the tasks.