Learn: OverfittingAndUnderfitting、一种缓解方式之决策回归树中设置max_leaf_nodes

过拟合、欠拟合

以决策树为例，来说。
dataset被划分到叶子，树太浅，假如数据集仅被split成了2个groups（划分的粒度特粗），每个group里必然特别多的houses。如果树特别地深，假如数据集被split成了1024个groups（划分的粒度特粗），叶子特别多，每个叶子上的houses则特别地少。
简单说，树太deep，易发生过拟合。
树太shallow，易发生欠拟合。

Overfitting: capturing spurious patterns that won’t recur in the future, leading to less accurate predictions, or
Underfitting: failing to capture relevant patterns, again leading to less accurate predictions.

overfitting：该模型capture了将来不会再出现的**“骗人模式”（spurious patterns）**，对训练数据拟合地perfect，But对验证集、对新数据的预测超级poor。
unfitting：该模型没有能capture到数据中 important distinctions and patterns。因为我们关注模型在新数据上的准确度，我们评估验证集，我们想找到过拟合和欠拟合之间的the sweet spot（最佳点）

一种解决：决策回归树中设置max_leaf_nodes

比如，在决策树中，我们可以设置max_leaf_nodes值。如，DecisionTreeRegressor函数提供了max_leaf_nodes。
尝试不同的max_leaf_nodes值，观察Error，找到最佳的max_leaf_nodes。

import pandas as pd
from sklearn.model_selection import train_test_split

#read csv
melbourne_data= pd.read_csv(r'G:\kaggle\melb_data.csv')
#drop NaN
filtered_melbourne_data= melbourne_data.dropna(axis=0)#pandas的 pd.dropna()

#target
y= filtered_melbourne_data.Price
#choosing features
melbourne_features=['Rooms', 'Bathroom', 'Landsize', 'BuildingArea', 'YearBuilt', 'Lattitude', 'Longtitude']
#X
X= filtered_melbourne_data[melbourne_features]

#split data into two pieces: training data and validation data
train_X, val_X, train_y, val_y = train_test_split(X, y, test_size=0.33, random_state=0 )#random_state设为0，重复实验室,其它参数不变时，每次得到的随机数组是不一样的。

from sklearn.tree import DecisionTreeRegressor
from sklearn.metrics import mean_absolute_error

#compare MAE scores from different values for max_leaf_nodes
def get_mae(max_leaf_nodes, train_X, val_X, train_y, val_y ):
    model= DecisionTreeRegressor(max_leaf_nodes= max_leaf_nodes, random_state=1)
    model.fit(train_X, train_y)
    val_prediction_y= model.predict(val_X)
    mae= mean_absolute_error(val_y, val_prediction_y)
    return mae

help(DecisionTreeRegressor)

#compare MAE with differing values of max_leaf_nodes
for max_leaf_nodes in [5,50,500,5000]:
    my_mae= get_mae(max_leaf_nodes,train_X, val_X, train_y, val_y )
    print("When max_leaf_nodes is ",max_leaf_nodes,", MAE is " ,my_mae)

('When max_leaf_nodes is ', 5, ', MAE is ', 345357.4675454862)
('When max_leaf_nodes is ', 50, ', MAE is ', 258991.08393549395)
('When max_leaf_nodes is ', 500, ', MAE is ', 244241.84628754357)
('When max_leaf_nodes is ', 5000, ', MAE is ', 253299.98630806847)

可见，500是最优的叶子节点数量。