java移植到spark_java – 无法导入spark包

我试图在Spark

JavaKafkaWordCount.java提供的示例Java代码中进行更改,并尝试使用Maven构建它.我正在使用Spark独立模块.

我正在使用以下pom.xml：

com.mycompany.app

my-app

4.0.0

Simple Project

jar

1.0

org.apache.spark

spark-core_2.10

1.5.1

junit

junit

4.8.2

test

org.apache.spark

spark-streaming-kafka_2.10

1.5.1

我跑的时候

mvn package

我收到以下错误：

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project my-app: Compilation failure: Compilation failure:

[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[17,34] package org.apache.spark.streaming does not exist

[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[18,43] package org.apache.spark.streaming.api.java does not exist

[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[19,43] package org.apache.spark.streaming.api.java does not exist

[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[20,43] package org.apache.spark.streaming.api.java does not exist

[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[21,43] package org.apache.spark.streaming.api.java does not exist

[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[39,5] cannot find symbol

[ERROR] symbol: class JavaStreamingContext

[ERROR] location: class com.mycompany.app.JavaKafkaWordCount

[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[39,37] cannot find symbol

[ERROR] symbol: class JavaStreamingContext

[ERROR] location: class com.mycompany.app.JavaKafkaWordCount

[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[39,73] cannot find symbol

[ERROR] symbol: class Duration

[ERROR] location: class com.mycompany.app.JavaKafkaWordCount

[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[48,5] cannot find symbol

[ERROR] symbol: class JavaPairReceiverInputDStream

[ERROR] location: class com.mycompany.app.JavaKafkaWordCount

[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[48,5] cannot find symbol

[ERROR] symbol: class JavaPairReceiverInputDStream

[ERROR] location: class com.mycompany.app.JavaKafkaWordCount

[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[49,23] cannot access org.apache.spark.streaming.dstream.ReceiverInputDStream

[ERROR] class file for org.apache.spark.streaming.dstream.ReceiverInputDStream not found

[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[51,5] cannot find symbol

[ERROR] symbol: class JavaDStream

[ERROR] location: class com.mycompany.app.JavaKafkaWordCount

[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[58,5] cannot find symbol

[ERROR] symbol: class JavaDStream

[ERROR] location: class com.mycompany.app.JavaKafkaWordCount

[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[65,5] cannot find symbol

[ERROR] symbol: class JavaPairDStream

[ERROR] location: class com.mycompany.app.JavaKafkaWordCount

[ERROR] -> [Help 1]

[ERROR]

[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.

[ERROR] Re-run Maven using the -X switch to enable full debug logging.

[ERROR]

[ERROR] For more information about the errors and possible solutions, please read the following articles:

[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

似乎我必须修改我的PATH变量,但我无法找到合适的jar.