JavaDay16

 学习来源：日撸 Java 三百行（61-70天，决策树与集成学习）_闵帆的博客——CSDN博客

集成学习之AdaBoosting

1.集成学习：

        集成学习是指通过构建并结合多个学习器来完成学习任务。

2.集成学习的结构：

        先产生一组个体学习器，再用某种策略将它们结合起来。   

图1.集成学习结构例子

        如图1所示，个体学习器一般为常见的机器学习算法，如：决策树，神经网络等。在同质集成中的个体学习器称为基学习器，其中同质的意思是个体学习器全是同一类型。结合模块通过将多个基学习器结合，通常会获得比单一学习器显著优越的泛化性能。

3.AdaBoosting算法：

        AdaBoosting算法是一个简单的弱分类算法提升过程，通过不断的训练，可以提高对数据的分类能力。过程如下：

        1）对初始训练数据进行权值分配；

        2）通过对带权训练数据的学习得到第一个弱分类器；

        3）根据前一个分类器的分类结果对数据的权值进行更新，然后通过对更新权值后的数据的学习得到一个新的弱分类器；

        4）重复 3）的操作得到第3，4，5 ... n个弱分类器；

        5）最终将n个弱分类器进行加权结合，即某个数据被分为哪一类要通过n个弱分类器的加权投票结果来决定。

4.带权数据集：

        初始时每条训练集数据拥有相同的权值。根据基分类器对数据的分类对错来更新对应数据的权值。当分类正确时，对应数据权值减少；反之分类错误时，对应数据权值增加。

5.代码如下：

package JavaDay16;

import weka.core.Instances;

import java.io.FileReader;
import java.util.Arrays;

/**
 * @author Ke-Xiong Wang
 *
 * Weighted instances.
 */
public class WeightedInstances extends Instances {

    /**
     * Just the requirement of some classes, any number is ok.
     */
    private static final long serialVersionUID = 11087456L;

    /**
     * Weights.
     */
    private double[] weights;

    /**
     ******************
     * The first constructor.
     *
     * @param paraFileReader
     *            The given reader to read data from file.
     ******************
     */
    public WeightedInstances(FileReader paraFileReader) throws Exception {
        super(paraFileReader);
        setClassIndex(numAttributes() - 1);

        // Initialize weights
        weights = new double[numInstances()];
        double tempAverage = 1.0 / numInstances();
        for (int i = 0; i < weights.length; i++) {
            weights[i] = tempAverage;
        } // Of for i
        System.out.println("Instances weights are: " + Arrays.toString(weights));
    } // Of the first constructor

    /**
     ******************
     * The second constructor.
     *
     * @param paraInstances
     *            The given instance.
     ******************
     */
    public WeightedInstances(Instances paraInstances) {
        super(paraInstances);
        setClassIndex(numAttributes() - 1);

        // Initialize weights
        weights = new double[numInstances()];
        double tempAverage = 1.0 / numInstances();
        for (int i = 0; i < weights.length; i++) {
            weights[i] = tempAverage;
        } // Of for i
        System.out.println("Instances weights are: " + Arrays.toString(weights));
    } // Of the second constructor

    /**
     ******************
     * Getter.
     *
     * @param paraIndex
     *            The given index.
     * @return The weight of the given index.
     ******************
     */
    public double getWeight(int paraIndex) {
        return weights[paraIndex];
    } // Of getWeight

    /**
     ******************
     * Adjust the weights.
     *
     * @param paraCorrectArray
     *            Indicate which instances have been correctly classified.
     * @param paraAlpha
     *            The weight of the last classifier.
     ******************
     */
    public void adjustWeights(boolean[] paraCorrectArray, double paraAlpha) {
        // Step 1. Calculate alpha.
        double tempIncrease = Math.exp(paraAlpha);

        // Step 2. Adjust.
        double tempWeightsSum = 0; // For normalization.
        for (int i = 0; i < weights.length; i++) {
            if (paraCorrectArray[i]) {
                weights[i] /= tempIncrease;
            } else {
                weights[i] *= tempIncrease;
            } // Of if
            tempWeightsSum += weights[i];
        } // Of for i

        // Step 3. Normalize.
        for (int i = 0; i < weights.length; i++) {
            weights[i] /= tempWeightsSum;
        } // Of for i

        System.out.println("After adjusting, instances weights are: " + Arrays.toString(weights));
    } // Of adjustWeights

    /**
     ******************
     * Test the method.
     ******************
     */
    public void adjustWeightsTest() {
        boolean[] tempCorrectArray = new boolean[numInstances()];
        for (int i = 0; i < tempCorrectArray.length / 2; i++) {
            tempCorrectArray[i] = true;
        } // Of for i

        double tempWeightedError = 0.3;

        adjustWeights(tempCorrectArray, tempWeightedError);

        System.out.println("After adjusting");

        System.out.println(toString());
    } // Of adjustWeightsTest

    /**
     ******************
     * For display.
     ******************
     */
    public String toString() {
        String resultString = "I am a weighted Instances object.\r\n" + "I have " + numInstances() + " instances and "
                + (numAttributes() - 1) + " conditional attributes.\r\n" + "My weights are: " + Arrays.toString(weights)
                + "\r\n" ;//+ "My data are: \r\n" + super.toString();

        return resultString;
    } // Of toString

    /**
     ******************
     * For unit test.
     *
     * @param args
     *            Not provided.
     ******************
     */
    public static void main(String args[]) {
        WeightedInstances tempWeightedInstances = null;
        String tempFilename = "d:/iris.arff";
        try {
            FileReader tempFileReader = new FileReader(tempFilename);
            tempWeightedInstances = new WeightedInstances(tempFileReader);
            tempFileReader.close();
        } catch (Exception exception1) {
            System.out.println("Cannot read the file: " + tempFilename + "\r\n" + exception1);
            System.exit(0);
        } // Of try

        System.out.println(tempWeightedInstances.toString());

        tempWeightedInstances.adjustWeightsTest();
    } // Of main
} // Of class WeightedInstances

运行结果：