AdaBoost实例

试用AdaBoost算法学习一个强分类器

训练数据集

序号	1	2	3	4	5	6	7	8	9	10
x	0	1	2	3	4	5	6	7	8	9
y	1	1	1	-1	-1	-1	1	1	1	-1

解：
初始化数据权值分布
$$\begin{gathered} D_1=(w_{1,1},w_{1,2},\dots ,w_{1,10}) \\ {w}_{1,i}=0.1,i=1,2,\dots ,10 \end{gathered}$$
对于$m=1$,
  (a)在权值分布为$D_1$的训练数据上，计算阈值$\nu$取[0.5,1.5,2.5,3.5,4.5,5.5,6.5,7.5,8.5]时分类误差率，

序号	1	2	3	4	5	6	7	8	9
$\nu$	0.5	1.5	2.5	3.5	4.5	5.5	6.5	7.5	8.5
分类误差率	0.5	0.4	0.3	0.4	0.5	0.4	0.5	0.4	0.3

阈值取$\nu =8.5$时分类误差率最低，故基本分类器为
$$G_1(x)=\{\begin{array}{ll}1,& x<8.5\\ -1,& x\ge 8.5\end{array}$$
  (b)$G_1(x)$在训练数据集上的误差率$e_1=P(G_1(x_i)̸=y_i)=0.3$
  ©计算$G_1(x)$的系数:${\alpha }_1=\frac{1}{2}log\frac{1-e_1}{e_1}=0.4236$
  (d)更新训练数据的权值分布：
$$D_2=(w_{2,1},w_{2,2},\dots ,w_{2,10})$$
$$w_{2,i}=\frac{w_{1,i}}{Z_1}exp(-{\alpha }_1{y}_i{G}_1(x_i)),i=1,2,\dots ,10$$
$$D_2=(0.07142857,0.07142857,0.07142857,0.16666667,0.16666667,0.16666667,0.07142857,0.07142857,0.07142857,0.07142857)$$
$$f_1(x)={\alpha }_1{G}_1(x)=0.4236G_1(x)$$
  (e)分类器$sign[f_1(x)]$在训练数据集上有3个误分点

序号	1	2	3	4	5	6	7	8	9	10
$G_1(x)$	1	1	1	1	1	1	1	1	1	-1
$f_1(x)$	0.4236	0.4236	0.4236	0.4236	0.4236	0.4236	0.4236	0.4236	0.4236	-0.4236
$sign[f_1(x)]$	1	1	1	1	1	1	1	1	1	-1
y	1	1	1	-1	-1	-1	1	1	1	-1

对$m=2$,
  (a)在权值分布为$D_2$的训练数据上，计算阈值$\nu$取[0.5,1.5,2.5,3.5,4.5,5.5,6.5,7.5,8.5]时分类误差率，$e_m={\sum }_{G_m(x_i)̸=y_i}{w}_{mi}$

序号	1	2	3	4	5	6	7	8	9
$\nu$	0.5	1.5	2.5	3.5	4.5	5.5	6.5	7.5	8.5
分类误差率	0.357	0.286	0.214	0.381	0.452	0.286	0.358	0.429	0.5

阈值取$\nu =2.5$时分类误差率最低，故基本分类器为
$$G_2(x)=\{\begin{array}{ll}1,& x<2.5\\ -1,& x\ge 2.5\end{array}$$
  (b)$G_2(x)$在训练数据集上的误差率$e_2=P(G_2(x_i)̸=y_i)=0.214$
  ©计算$G_2(x)$的系数:${\alpha }_2=\frac{1}{2}log\frac{1-e_2}{e_2}=0.6496$
  (d)更新训练数据的权值分布：
$$D_3=(w_{3,1},w_{3,2},\dots ,w_{3,10})$$
$$w_{3,i}=\frac{w_{2,i}}{Z_1}exp(-{\alpha }_2{y}_i{G}_2(x_i)),i=1,2,\dots ,10$$
$$D_3=(0.04545452,0.04545452,0.04545452,0.10606056,0.10606056,0.10606056,0.16666675,0.16666675,0.16666675,0.04545452)$$
$$f_2(x)=0.4236G_1(x)+0.6496G_2(x)$$
  (e)分类器$sign[f_2(x)]$在训练数据集上有3个误分点

序号	1	2	3	4	5	6	7	8	9	10
$G_1(x)$	1	1	1	1	1	1	1	1	1	-1
$G_2(x)$	1	1	1	-1	-1	-1	-1	-1	-1	-1
${\alpha }_1{G}_1(x)$	0.4236	0.4236	0.4236	0.4236	0.4236	0.4236	0.4236	0.4236	0.4236	-0.4236
${\alpha }_2{G}_2(x)$	0.6496	0.6496	0.6496	-0.6496	-0.6496	-0.6496	-0.6496	-0.6496	-0.6496	-0.6496
$sign[f_2(x)]$	1	1	1	-1	-1	-1	-1	-1	-1	-1
y	1	1	1	-1	-1	-1	1	1	1	-1

对$m=3$
  (a)在权值分布为$D_3$的训练数据上，计算阈值$\nu$取[0.5,1.5,2.5,3.5,4.5,5.5,6.5,7.5,8.5]时分类误差率，$e_m={\sum }_{G_m(x_i)̸=y_i}{w}_{mi}$

序号	1	2	3	4	5	6	7	8	9
$\nu$	0.5	1.5	2.5	3.5	4.5	5.5	6.5	7.5	8.5
分类误差率	0.409	0.455	0.5	0.394	0.288	0.182	0.348	0.485	0.318

阈值取$\nu =5.5$时分类误差率最低，故基本分类器为
$$G_2(x)=\{\begin{array}{ll}-1,& x<5.5\\ 1,& x\ge 5.5\end{array}$$
  (b)$G_3(x)$在训练数据集上的误差率$e_3=P(G_3(x_i)̸=y_i)=0.7520$
  (d)更新训练数据的权值分布：
$$D_4=(w_{4,1},w_{4,2},\dots ,w_{4,10})$$
$$w_{4,i}=\frac{w_{3,i}}{Z_1}exp(-{\alpha }_3{y}_i{G}_3(x_i)),i=1,2,\dots ,10$$
$$D_4=(0.125,0.125,0.125,0.06481478,0.06481478,0.06481478,0.10185189,0.10185189,0.10185189,0.125)$$
$$f_3(x)=0.4236G_1(x)+0.6496G_2(x)+0.7520G_3(x)$$
  (e)分类器$sign[f_3(x)]$在训练数据集上有0个误分点

序号	1	2	3	4	5	6	7	8	9	10
$G_1(x)$	1	1	1	1	1	1	1	1	1	-1
$G_2(x)$	1	1	1	-1	-1	-1	-1	-1	-1	-1
$G_3(x)$	-1	-1	-1	-1	-1	-1	1	1	1	1
${\alpha }_1{G}_1(x)$	0.4236	0.4236	0.4236	0.4236	0.4236	0.4236	0.4236	0.4236	0.4236	-0.4236
${\alpha }_2{G}_2(x)$	0.6496	0.6496	0.6496	-0.6496	-0.6496	-0.6496	-0.6496	-0.6496	-0.6496	-0.6496
${\alpha }_3{G}_3(x)$	-0.7520	-0.7520	-0.7520	-0.7520	-0.7520	-0.7520	0.7520	0.7520	0.7520	0.7520
$sign[f_3(x)]$	1	1	1	-1	-1	-1	1	1	1	-1
y	1	1	1	-1	-1	-1	1	1	1	-1

于是最终的分类器为
$G(x)=sign[f_3(x)]=0.4236G_1(x)+0.6496G_2(x)+0.7520G_3(x)$