Basics
ID3 Algorithm
Give the wiki link: ID3 Algorithm in Wiki
Information Gain
Entropy:
H=−∑i=0np(xi)log2p(xi)
The higher the entropy, the more mixed up the data is.
We can also have a basic knowledge about the Gini impurity.