spark 官网例子 统计一篇文章包含字母a的个数

代码： 

import org.apache.spark.api.java.*;
import org.apache.spark.SparkConf;
import org.apache.spark.api.java.function.Function;


public class SimpleApp {
    public static void main(String[] args) {
        SparkConf conf = new SparkConf().setAppName("Simple Application");
        JavaSparkContext sc = new JavaSparkContext(conf);
        JavaRDD<String> logData = sc.textFile(args[0]).cache();


        long numAs = logData.filter(new Function<String, Boolean>() {
            public Boolean call(String s) {
                return s.contains("a");
            }
        }).count();


        long numBs = logData.filter(new Function<String, Boolean>() {
            public Boolean call(String s) {
                return s.contains("b");
            }
        }).count();
        System.out.println("File contains a=" + numAs + ", and contains b=" + numBs);
    }

}

2、运行程序

spark-submit --master local --class com.test.spark.test.SimpleApp   spark-0.0.1.jar /user/cloudera/examples/src/org/apache/oozie/example/DemoPigMain.java

3、验证 使用 linux 命令验证结果

 grep 'b' /home/cloudera/caojian/sparkFile/wc.txt  | wc -l