1、整个实现的伪代码python:
对每次迭代:
利用buildstump()函数(上一篇提到过)找到最佳的单层决策树;
计算alpha(利用该决策树的错误率进行计算);
计算新的每个样本的权重D(使用alpha进行计算,第一次分对的样本,权重减小反之,权重增大)
给最佳单侧决策树的预测结果添加alpha权重(弱分类器(这里指最佳的单侧决策树)的错误率越小,赋予该分 类器的权重值alpha值就越大);
计算错误率(当错误率为0时,迭代停止!);
2、python代码:
def adaBoostTrainDS(dataArr,classLabels,numIt=40):
weakClassArr = []
m = shape(dataArr)[0]
D = mat(ones((m,1))/m) #init D to all equal
aggClassEst = mat(zeros((m,1)))
for i in range(numIt):
bestStump,error,classEst = buildStump(dataArr,classLabels,D)#build Stump
#print "D:",D.T
alpha = float(0.5*log((1.0-error)/max(error,1e-16)))#calc alpha, throw in max(error,eps) to account for error=0
bestStump['alpha'] = alpha
weakClassArr.append(bestStump) #store Stump Params in Array
#print "classEst: ",classEst.T
expon = multiply(-1*alpha*mat(classLabels).T,classEst) #exponent for D calc, getting messy
D = multiply(D,exp(expon)) #Calc New D for next iteration
D = D/D.sum()
#calc training error of all classifiers, if this is 0 quit for loop early (use break)
aggClassEst += alpha*classEst
#print "aggClassEst: ",aggClassEst.T
aggErrors = multiply(sign(aggClassEst) != mat(classLabels).T,ones((m,1)))
errorRate = aggErrors.sum()/m
print "total error: ",errorRate
if errorRate == 0.0: break
return weakClassArr,aggClassEst
3、输出结果
可以看出图中的结果包含三部字典,即包含三个弱分类器,使得最终错误率为0;可以仔细观察D因为上一次的结果权重改变