IDEA版本为2016.3
书写为简单的Spark WordCount代码
import org.apache.spark.SparkConf;
import org.apache.spark.api.java.JavaRDD;
import org.apache.spark.api.java.JavaSparkContext;
/**
* Created by Joe on 2017/4/20.
*/
public class WordCount {
public static void main(String[] args) {
SparkConf conf = new SparkConf().setMaster("local").setAppName("wordcount");
JavaSparkContext context = new JavaSparkContext(conf);
JavaRDD lineRdd = context.textFile("C:\\spark-2.1.0-bin-hadoop2.6\\README.md");
lineRdd.foreach(line -> System.out.println(line));
JavaRDD words = lineRdd.map(line -> {
String[] speces = line.split(" ");
return speces[0];
});
words.foreach(line -> System.out.println(line));
}
}
但是在IDEA里面会有代码提示报错
但是在程序编译的时候是没有问题的。代码可以正常运行。
代码是正常退出的。
其中的Pom文件为
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
4.0.0
org.learn
spark_maven
1.0-SNAPSHOT
nd4j-native-platform
UTF-8
bin
2.1.0
2.11.8
2.8.2
1.2.14
9.2.5.v20141112
2.17
1.8
org.apache.spark
spark-core_2.11
${spark.version}
org.apache.spark
spark-sql_2.10
${spark.version}
org.apache.spark
spark-mllib_2.10
${spark.version}
redis.clients
jedis
${jedis.version}
com.alibaba
fastjson
${fastjson.version}
org.eclipse.jetty
jetty-server
${jetty.version}
org.eclipse.jetty
jetty-servlet
${jetty.version}
org.eclipse.jetty
jetty-util
${jetty.version}
org.glassfish.jersey.core
jersey-server
${container.version}
org.glassfish.jersey.containers
jersey-container-servlet-core
${container.version}
org.glassfish.jersey.containers
jersey-container-jetty-http
${container.version}
org.glassfish.jersey.media
jersey-media-moxy
${container.version}
org.scala-lang
scala-compiler
${scala.version}
org.scala-lang
scala-reflect
${scala.version}
org.scala-lang
scala-library
${scala.version}
org.scala-lang
scala-actors
${scala.version}
org.scala-lang.modules
scala-parser-combinators_2.11
1.0.4
org.scala-lang.modules
scala-xml_2.11
1.0.2
org.scala-lang
scalap
${scala.version}
com.fasterxml.jackson.module
jackson-module-jaxb-annotations
2.7.5
com.fasterxml.jackson.core
jackson-annotations
2.7.5
org.codehaus.plexus
plexus-compiler-eclipse
2.2
org.apache.maven.plugins
maven-compiler-plugin
3.5
${java.version}
${java.version}
eclipse
请教如何使得IDEA不进行这个代码错误提示。