依赖冲突管理
ThisBuild / scalaVersion := "2.11.11"
ThisBuild / organization := "com.wtx.job014"
libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-sql" % "2.3.2" % "provided",
"org.apache.spark" %% "spark-core" % "2.3.2" % "provided",
"org.apache.spark" %% "spark-streaming" % "2.3.2" % "provided",
"org.apache.kafka" %% "kafka" % "2.0.0",
"org.apache.spark" %% "spark-streaming-kafka-0-10" % "2.3.2",
"redis.clients" % "jedis" % "3.2.0",
"commons-codec" % "commons-codec" % "1.14",
)
assemblyMergeStrategy in assembly := {
case PathList("org","apache","spark","unused","UnusedStubClass.class") => MergeStrategy.discard
case x =>
val oldStrategy = (assemblyMergeStrategy in assembly).value
oldStrategy(x)
}
当出现java版本不匹配时,使用libraryDependencies ++= Seq()
管理版本
当出现明明存在某个版本的依赖,但是sbt就是找不到的话,提示你远程仓库中没有此依赖url
那么就使用maven管理,更新一下这个版本的包,让他eclipse加载构建workspace,构建好后,maven会把依赖放到.m2本地依赖仓库
这个时候使用sbt assembly reload eclipse等指令都可以找到本地库中的依赖了~