Adaboost模型公式的参数推导

  假设最后根据各个基模型$G_i(x)i\in [1,m]$,各个基模型重要程度为${\alpha }_i$加权得到的模型为$f_m(x)$,其中 y ∈ { − 1 , 1 } y\in{\{-1,1\}} y∈{−1,1}

$$f_m(x)=\sum _{i=1}^m{\alpha }_i{G}_i(x)$$
$$f_m(x)=\sum _{i=1}^{m-1}{\alpha }_i{G}_i(x)+{\alpha }_m{G}_m(x)$$
$$f_m(x)=f_{m-1}(x)+{\alpha }_m{G}_m(x)$$

  基学习器的损失函数为$L(y,f(x))=e^{-yf(x)}$

  所以整个Adaboost模型的损失函数为：
$$L=\sum _{i=1}^{n}exp(-y_{i}f(x_i))$$

  该损失函数的${\alpha }_m$和$G_m(x)$是需要求得的

$$({\alpha }_m,G_m(x))=\underset{{\alpha }_m,G_m}{\mathrm{argmin}}\sum _{i=1}^{n}exp(-y_i(f_{m{-}_1}(x_i)+{\alpha }_m{G}_m(x_i)))$$

其中${\sum }_{i=1}^{n}exp(-y_i(f_{m-1}(x_i)+{\alpha }_m{G}_m(x_i))={\sum }_{i=1}^{n}exp(-y_i{f}_{m-1}(x_i))exp(-y_i{\alpha }_m{G}_m(x_i))$–(1)

  设${\omega }_i^m=exp(-y_i{f}_{m-1}(x_i))$,将其带入式(1)得到：

$\sum _{i=1}^n{\omega }_i^{m}exp(-y_i{\alpha }_m{G}_m(x_i))$–(2)

  当$y_i=G_m(x_i)$时，$y_i{G}_m(x_i)=1$,当$y_i\ne G_m(x_i)$时，$y_i{G}_m(x_i)=-1$,所以式(2)可以写成

  $\sum _{y_i=G_m(x_i)}^n{\omega }_i^{m}exp(-{\alpha }_m)+\sum _{y_i\ne G_m(x_i)}^n{\omega }_i^{m}exp({\alpha }_m)$

  $=\sum _{y_i=G_m(x_i)}^n{\omega }_i^{m}exp(-{\alpha }_m)+\sum _{y_i\ne G_m(x_i)}^n{\omega }_i^{m}exp({\alpha }_m)+\sum _{y_i\ne G_m(x_i)}^n{\omega }_i^{m}exp(-{\alpha }_m)-\sum _{y_i\ne G_m(x_i)}^n{\omega }_i^{m}exp(-{\alpha }_m)$

  $=e^{-{\alpha }_m}{\sum }_{i=1}^n{\omega }_i^m-(e^{-{\alpha }_m}-e^{{\alpha }_m})\sum _{y_i\ne G_m(x_i)}^n{\omega }_i^m$

  $=e^{-{\alpha }_m}\sum _{i=1}^n{\omega }_i^m-(e^{-{\alpha }_m}-e^{{\alpha }_m})\sum _{i=1}^n{\omega }_i^{m}I(y_i\ne G_m(x_i))$–(3)

  要使式(3)最小，则$G_m(x)$应该取$\underset{G}{\mathrm{argmin}}\sum _{i=1}^n{\omega }_i^{m}I(y_i\ne G_m(x_i))$

  因为每次求每个点的重要程度时都会除以总和，所以$\sum _{i=1}^n{\omega }_i^m=1$，对式(3)中的${\alpha }_m$求导得到：

  $-e^{-{\alpha }_m}-(-e^{-{\alpha }_m}-e^{{\alpha }_m})\sum _{i=1}^n{\omega }_i^{n}I(y_i\ne G_m(x_i))=0$

  $e^{-{\alpha }_m}(1-\sum _{i=1}^n{\omega }_i^{n}I(y_i\ne G_m(x_i)))=e^{{\alpha }_m}\sum _{i=1}^{n}I(y_i\ne G_m(x_i))$–(4)

  令$\sum _{i=1}^n{\omega }_{i=1}^{n}I(y_i\ne G_m(x_i))=e_m$

  化简式(4)得到${\alpha }_m=\frac{1}{2}ln(\frac{1-e_m}{e_m})$

  因为${\omega }_i^m=exp(-y_i{f}_{m-1}(x_i))$可以推导得

${\omega }_i^{m+1}=exp(-y_i{f}_m(x_i))=exp(-y_i(f_{m-1}(x_i)+{\alpha }_m{G}_m(x_i)))=exp(-y_i{f}_{m-1})exp(-y_i{\alpha }_m{G}_m(x_i))$

$={\omega }_i^{m}exp(-y_i{\alpha }_m{G}_m(x_i))$