# 统计学习方法笔记（十二）提升方法（二）

## 提升树

1、提升树模型

${f}_{M}\left(x\right)=\sum _{m=1}^{M}T\left(x;{\mathrm{\Theta }}_{m}\right)$${f_M}(x) = \sum\limits_{m = 1}^M {T(x;{\Theta _m})}$

2、提升树算法

${f}_{m}\left(x\right)={f}_{m-1}\left(x\right)+T\left(x;{\mathrm{\Theta }}_{m}\right)$${f_m}(x) = {f_{m - 1}}(x) + T(x;{\Theta _m})$

${\stackrel{^}{\mathrm{\Theta }}}_{m}=\mathrm{arg}\underset{{\mathrm{\Theta }}_{m}}{min}\sum _{i=1}^{N}L\left({y}_{i},{f}_{m-1}\left({x}_{i}\right)+T\left({x}_{i};{\mathrm{\Theta }}_{m}\right)\right)$${\widehat \Theta _m} = \arg \mathop {\min }\limits_{{\Theta _m}} \sum\limits_{i = 1}^N {L({y_i},{f_{m - 1}}({x_i}) + T({x_i};{\Theta _m})} )$
（1）回归问题的提升树

$T\left(x;{\mathrm{\Theta }}_{m}\right)=\sum _{j=1}^{J}{c}_{j}I\left(x\in {R}_{j}\right)$$T(x;{\Theta _m}) = \sum\limits_{j = 1}^J {{c_j}I(x \in {R_j})}$

$\begin{array}{l} L(y,{f_{m - 1}}(x) + T(x;{\Theta _m}))\ = {[y - {f_{m - 1}}(x) - T(x;{\Theta _m})]^2}\ = {[r - T(x;{\Theta _m})]^2} \end{array}$

3、梯度提升

（1）初始化：${f}_{0}\left(x\right)=\mathrm{arg}\underset{c}{min}\sum _{i=1}^{N}L\left({y}_{i},c\right)$${f_0}(x) = \arg \mathop {\min }\limits_c \sum\limits_{i = 1}^N {L({y_i},c)}$
（2）计算残差：${r}_{mi}=-{\left[\frac{\mathrm{\partial }L\left(y,f\left({x}_{i}\right)\right)}{\mathrm{\partial }f\left({x}_{i}\right)}\right]}_{f\left(x\right)={f}_{m-1}\left(x\right)}$${r_{mi}} = - {\left[ {\frac{{\partial L(y,f({x_i}))}}{{\partial f({x_i})}}} \right]_{f(x) = {f_{m - 1}}(x)}}$
（3）对残差拟合回归树
（4）计算：${c}_{mj}=\mathrm{arg}\underset{c}{min}\sum _{{x}_{i}\in {R}_{mj}}L\left({y}_{i},{f}_{m-1}\left({x}_{i}\right)+c\right)$${c_{mj}} = \arg \mathop {\min }\limits_c \sum\limits_{{x_i} \in {R_{mj}}} {L({y_i},{f_{m - 1}}({x_i}) + c)}$
（5）更新${f}_{m}\left(x\right)={f}_{m-1}\left(x\right)+\sum _{j=1}^{J}{c}_{mj}I\left(x\in {R}_{mj}\right)$${f_m}(x) = {f_{m - 1}}(x) + \sum\limits_{j = 1}^J {{c_{mj}}I(x \in {R_{mj}})}$
（6）得到回归树$\stackrel{^}{f}\left(x\right)={f}_{M}\left(x\right)=\sum _{m=1}^{M}\sum _{j=1}^{J}{c}_{mj}I\left(x\in {R}_{mj}\right)$$\widehat f(x) = {f_M}(x) = \sum\limits_{m = 1}^M {\sum\limits_{j = 1}^J {{c_{mj}}I(x \in {R_{mj}})} }$

• 广告
• 抄袭
• 版权
• 政治
• 色情
• 无意义
• 其他

120