spark idea java.lang.NoClassDefFoundError

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/fs/FSDataInputStream
	at org.apache.spark.SparkConf.loadFromSystemProperties(SparkConf.scala:73)
	at org.apache.spark.SparkConf.<init>(SparkConf.scala:68)
	at org.apache.spark.SparkConf.<init>(SparkConf.scala:55)

也可能是

Exception in thread "main" java.lang.NoClassDefFoundError: scala/Cloneable 
at java.lang.ClassLoader.defineClass1(Native Method) 
at java.lang.ClassLoader.defineClass(ClassLoader.java

都是类似的报错,这个主要是maven依赖的scope设置不对 ,注释掉<scope>provided</scope>

    <dependency>
      <groupId>org.apache.spark</groupId>
      <artifactId>spark-core_${scala.binary.version}</artifactId>
      <version>${project.version}</version>
     <!-- <scope>provided</scope> -->
    </dependency>

一般是再idea 运行spark 的 mainclass 报错

©️2020 CSDN 皮肤主题: 大白 设计师:CSDN官方博客 返回首页