机器学习----xgboost学习笔记

1、利用xgboost做特征组合
1）XGBModel.apply(self, X, ntree_limit=0)
return the predicted leaf every tree for each sample
X: 训练集特征，features matrix
ntree_limit: 预测时数的个数， Limit number of trees in the prediction; defaults to 0 (use all trees)。

    def apply(self, X, ntree_limit=0):
        """Return the predicted leaf every tree for each sample.

        Parameters
        ----------
        X : array_like, shape=[n_samples, n_features]
            Input features matrix.

        ntree_limit : int
            Limit number of trees in the prediction; defaults to 0 (use all trees).

        Returns
        -------
        X_leaves : array_like, shape=[n_samples, n_trees]
            For each datapoint x in X and for each tree, return the index of the
            leaf x ends up in. Leaves are numbered within
            ``[0; 2**(self.max_depth+1))``, possibly with gaps in the numbering.
        """
        test_dmatrix = DMatrix(X, missing=self.missing)
        return self.get_booster().predict(test_dmatrix,
                                          pred_leaf=True,
                                          ntree_limit=ntree_limit)

1. GBDT与GBDT+LR区别
   我理解如下：
   GBDT： 拟合上次预测后与实际结果的残差（即拟合[(y-y1^)-y^2]）.
   GBDT+LR: 即将GBDT每课树预测的结果，通过线性再次组合，自动学习每次的权重。

菜鸟学习中，笔记方便自己后期学习理解，边学习边修改中，如有不正确的地方，烦请指正。