IDEA下sparkMLlib出现java.lang.AbstractMethodError错误

最新推荐文章于 2024-07-17 13:55:08 发布

liuhehe123

最新推荐文章于 2024-07-17 13:55:08 发布

阅读量691

点赞数

分类专栏： spark Scala 大数据文章标签： sparkMLlib spark

本文链接：https://blog.csdn.net/liuhehe123/article/details/88050233

版权

大数据同时被 3 个专栏收录

16 篇文章 0 订阅

订阅专栏

spark

11 篇文章 0 订阅

订阅专栏

Scala

8 篇文章 0 订阅

订阅专栏

在写sparkMLlib程序的时候出现 java.lang.AbstractMethodError错误，at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)

。。。balabalabala

这个一般是包的版本出现了问题，

我当时用的包 spark-mllib 2.1.0 改成 2.3.0错误解决。

libraryDependencies += "org.apache.spark" % "spark-mllib_2.11" % "2.3.0"

build.sbt具体内容容下：

name := "Simple Project"
version := "1.0"
scalaVersion := "2.11.8"
libraryDependencies += "org.apache.spark" % "spark-core_2.11" % "2.1.0"
libraryDependencies += "org.apache.hbase" % "hbase-client" % "1.1.2"
libraryDependencies += "org.apache.hbase" % "hbase-common" % "1.1.2"
libraryDependencies += "org.apache.hbase" % "hbase-server" % "1.1.2"
//libraryDependencies += "org.apache.spark" % "spark-sql" % "8.0.15"
libraryDependencies += "org.apache.spark" %% "spark-sql" % "2.2.0"
libraryDependencies += "mysql" % "mysql-connector-java" % "8.0.15"
libraryDependencies += "org.apache.spark" %% "spark-hive" % "2.3.0"
libraryDependencies += "org.apache.spark" % "spark-mllib_2.11" % "2.3.0"