SparkMLlib Java 决策树分类算法(DecisionTree)

决策树基本理解：

决策树利用树形结构，根据特征一层一层做出判断，会在某一层得到结果。我在其他博客中看到了一副非常好的诠释图：

SparkMLlib Java程序所用数据：

   训练数据：C:\hello\trainData.txt

该数据，逗号前为目标向量，逗号后为特征向量(空格隔开)。

   测试数据：C:\hello\testData.txt

该数据为特征向量，空格隔开。

SparkMLlib DecisionTreeJava程序：

package MLlibTest;

import java.util.HashMap;
import java.util.Map;

import org.apache.spark.SparkConf;
import org.apache.spark.api.java.JavaPairRDD;
import org.apache.spark.api.java.JavaRDD;
import org.apache.spark.api.java.JavaSparkContext;
import org.apache.spark.api.java.function.Function;
import org.apache.spark.api.java.function.PairFunction;
import org.apache.spark.api.java.function.VoidFunction;
import org.apache.spark.mllib.linalg.Vector;
import org.apache.spark.mllib.linalg.Vectors;
import org.apache.spark.mllib.regression.LabeledPoint;
import org.apache.spark.mllib.tree.DecisionTree;
import org.apache.spark.mllib.tree.model.DecisionTreeModel;

import scala.Tuple2;

public class DecisionTreeTest{

	public static void main(String[] args) {
		 SparkConf conf = new SparkConf().setAppName("DecisionTreeTest").setMaster("local[*]");
 	     JavaSparkContext jsc = new JavaSparkContext(conf);
 	     JavaRDD<String> lines = jsc.textFile("C://hello//trainData.txt");
 	     JavaRDD<LabeledPoint> transdata = lines.map(new Function<String,LabeledPoint>(){
 	    	 private static final long serialVersionUID = 1L;
			 @Override	 
 	    	public LabeledPoint call(String str) throws Exception{
				 String[] t1 = str.split(",");
					String[] t2 = t1[1].split(" ");
					LabeledPoint lab = new LabeledPoint(Double.parseDouble(t1[0]),
							Vectors.dense(Double.parseDouble(t2[0]), Double.parseDouble(t2[1]), Double.parseDouble(t2[2])));
				return lab;
			}
 	     });
 	     //设置决策树参数，训练模型
 	    Integer numClasses = 3;
        Map<Integer, Integer> categoricalFeaturesInfo = new HashMap<Integer, Integer>();
        String impurity = "gini";
        Integer maxDepth = 5;
        Integer maxBins = 32;
        final DecisionTreeModel tree_model = DecisionTree.trainClassifier(transdata, numClasses,categoricalFeaturesInfo, impurity, maxDepth, maxBins);
        System.out.println("决策树模型：");  
        System.out.println(tree_model.toDebugString());
        //保存模型
        tree_model.save(jsc.sc(), "C://hello//DecisionTreeModel");
        
        
        //未处理数据，带入模型处理
        JavaRDD<String> testLines = jsc.textFile("C://hello//testData.txt");
        JavaPairRDD<String,String> res = testLines.mapToPair(new PairFunction<String, String, String>() {
        	private static final long serialVersionUID = 1L;
        	@Override
        	public Tuple2<String,String> call(String line) throws Exception{
        		String[] t2 = line.split(" ");
				Vector v = Vectors.dense(Double.parseDouble(t2[0]), Double.parseDouble(t2[1]),
						Double.parseDouble(t2[2]));
				double res = tree_model.predict(v);
        		return new Tuple2<String,String>(line,Double.toString(res));
        	}
		}).cache();
        //打印结果
 	    res.foreach(new VoidFunction<Tuple2<String,String>>() {
 	    	private static final long serialVersionUID = 1L;
			 @Override	 
	    	public void call(Tuple2<String,String> a) throws Exception{
				 System.out.println(a._1+" : "+a._2);
			}
		});
 	    //将结果保存在本地
 	    res.saveAsTextFile("C://hello/res");
	}

}

结语：

     做的时间匆忙，错误之处，请大家指出批评，相互学习。