spark 运行java程序_如何运行spark java程序

结合从官方

Quick Start Guide和

Launching Spark on YARN的步骤，我们得到：

我们将创建一个非常简单的Spark应用程序SimpleApp.java：

/*** SimpleApp.java ***/

import org.apache.spark.api.java.*;

import org.apache.spark.api.java.function.Function;

public class SimpleApp {

public static void main(String[] args) {

String logFile = "$YOUR_SPARK_HOME/README.md"; // Should be some file on your system

JavaSparkContext sc = new JavaSparkContext("local", "Simple App",

"$YOUR_SPARK_HOME", new String[]{"target/simple-project-1.0.jar"});

JavaRDD logData = sc.textFile(logFile).cache();

long numAs = logData.filter(new Function() {

public Boolean call(String s) { return s.contains("a"); }

}).count();

long numBs = logData.filter(new Function() {