梯度提升树GBDT帮助理解

梯度提升树GBDT

关键是利用损失函数的负梯度在当前模型的值作为回归问题提升树算法中的残差的近似值，拟合一个回归树。

$$-\left[\frac{\partial L(y,f(x_i))}{\partial f(x_i)}\right]$$
讲解GBDT的一个不错的博客

结合下面帮助理解
$$learnrate:0.1\Rightarrow 1.475-0.1\times 0.375=1.4375$$

index	id	age	weight	label	f_0	res_1	f_1	res_2	f_2	res_3	f_3	res_4	f_4	res_5	f_5
0	1	5	20	1.1	1.475	-0.375	1.4375	-0.3375	1.40375	-0.30375	1.373375	-0.273375	1.346037	-0.246037	1.321434
1	2	7	30	1.3	1.475	-0.175	1.4575	-0.1575	1.44175	-0.14175	1.427575	-0.127575	1.414818	-0.114818	1.403336
2	3	21	70	1.7	1.475	0.225	1.4975	0.2025	1.51775	0.18225	1.535975	0.164025	1.552377	0.147622	1.567140
3	4	30	60	1.8	1.475	0.325	1.5075	0.2925	1.53675	0.26325	1.563075	0.236925	1.586768	0.213232	1.608091

$$learnrate:0.2\Rightarrow 1.475-0.2\times 0.375=1.4$$

index	id	age	weight	label	f_0	res_1	f_1	res_2	f_2	res_3	f_3	res_4	f_4	res_5	f_5
0	1	5	20	1.1	1.475	-0.375	1.40	-0.30	1.340	-0.240	1.2920	-0.1920	1.25360	-0.15360	1.222880
1	2	7	30	1.3	1.475	-0.175	1.44	-0.14	1.412	-0.112	1.3896	-0.0896	1.37168	-0.07168	1.357344
2	3	21	70	1.7	1.475	0.225	1.52	0.18	1.556	0.144	1.5848	0.1152	1.60784	0.09216	1.626272
3	4	30	60	1.8	1.475	0.325	1.54	0.26	1.592	0.208	1.6336	0.1664	1.66688	0.13312	1.693504