Deciesion Tree is the foundation of the random forest.
A decision tree is a decision support tool that uses a tree-like model of decisions and their possible consequences, including chance event outcomes, resource costs, and utility. It is one way to display an algorithm that only contains conditional control statements.
Base on the history samples and experiences, we can build different decision tree(model), we defined a method to evaluate the performance of the decision tree.
Decision boundary of Decision Tree
The structure of the decision is complex. Structured Prediction
NP-hard problem
We can't solve a problem within the polynomial complexity called NP-hard
We use the approximate approach to solve the problem such as greedy algorithm.
Information Gain
Type 1 of Decision Tree
Type two of the decision tree
we change the order of the two features to obtain a new decision tree.
if the dimension of the input data is large, the number associate decision trees is very large!
We use the greedy algorithm to obtain the decision tree model.
The first step is to choose the root