应用场景一:统计单词总长度,以及类似场景
public class HelloWorldMap implements Serializable {
public static void main(String[] args) {
SparkConf sparkConf = new SparkConf().setAppName("HelloWorldMap").setMaster("local");
JavaSparkContext sc = new JavaSparkContext(sparkConf);
JavaRDD<String> lines = sc.textFile("/Users/ccc/Documents/data.txt");
JavaRDD<Integer> lineLengths = lines.map(line -> line.length());
lineLengths.persist(StorageLevel.MEMORY_ONLY());
lineLengths.foreach(lineLength -> System.out.println(lineLength));
Integer totalLength = lineLengths.reduce((lineLengthA, lineLengthB) -> lineLengthA + lineLengthB);
System.out.println(totalLength);
}
}
应用场景二:统计所有单词以及对应个数,获取统计结果,以及类似场景
public