(*:update) Conflicting cross-version suffixes in: org.scala-lang.modules:scala-xml

SBT异常信息如下:

[info] Loading project definition from D:\IdeaProjects\spark-practice\project
[info] Set current project to spark-practice (in build file:/D:/IdeaProjects/spark-practice/)
> [info] Defining *:shellPrompt
[info] The new value will be used by no settings or tasks.
[info] Reapplying settings...
[info] Set current project to spark-practice (in build file:/D:/IdeaProjects/spark-practice/)
[info] Defining */*:sbtStructureOutputFile
[info] The new value will be used by no settings or tasks.
[info] Reapplying settings...
[info] Set current project to spark-practice (in build file:/D:/IdeaProjects/spark-practice/)
[info] Defining */*:sbtStructureOptions
[info] The new value will be used by no settings or tasks.
[info] Reapplying settings...
[info] Set current project to spark-practice (in build file:/D:/IdeaProjects/spark-practice/)
[info] Applying State transformations org.jetbrains.sbt.CreateTasks from C:/Users/gyg/.IntelliJIdea2016.3/config/plugins/Scala/launcher/sbt-structure-0.13.jar
[info] Reapplying settings...
[info] Set current project to spark-practice (in build file:/D:/IdeaProjects/spark-practice/)
[error] Modules were resolved with conflicting cross-version suffixes in {file:/D:/IdeaProjects/spark-practice/}spark-practice:
[error]    org.scala-lang.modules:scala-xml _2.12, _2.11
[info] Resolving org.scala-sbt#sbt;0.13.15 ...
[info] Resolving org.scala-lang#scala-library;2.10.6 ...
[info] Resolving org.scala-sbt#main;0.13.15 ...
[info] Resolving org.scala-sbt#actions;0.13.15 ...
[info] Resolving org.scala-sbt#classpath;0.13.15 ...
[info] Resolving org.scala-lang#scala-compiler;2.10.6 ...
[info] Resolving org.scala-lang#scala-reflect;2.10.6 ...
[info] Resolving org.scala-sbt#interface;0.13.15 ...
[info] Resolving org.scala-sbt#io;0.13.15 ...
[info] Resolving org.scala-sbt#control;0.13.15 ...
[info] Resolving org.scala-sbt#launcher-interface;1.0.1 ...
[info] Resolving org.scala-sbt#completion;0.13.15 ...
[info] Resolving org.scala-sbt#collections;0.13.15 ...
[info] Resolving jline#jline;2.14.3 ...
[info] Resolving org.scala-sbt#api;0.13.15 ...
[info] Resolving org.scala-sbt#classfile;0.13.15 ...
[info] Resolving org.scala-sbt#logging;0.13.15 ...
[info] Resolving org.scala-sbt#process;0.13.15 ...
[info] Resolving org.scala-sbt#compiler-integration;0.13.15 ...
[info] Resolving org.scala-sbt#incremental-compiler;0.13.15 ...
[info] Resolving org.scala-sbt#relation;0.13.15 ...
[info] Resolving org.scala-sbt#compile;0.13.15 ...
[info] Resolving org.scala-sbt#persist;0.13.15 ...
[info] Resolving org.scala-tools.sbinary#sbinary_2.10;0.4.2 ...
[info] Resolving org.scala-sbt#compiler-ivy-integration;0.13.15 ...
[info] Resolving org.scala-sbt#ivy;0.13.15 ...
[info] Resolving org.scala-sbt#cross;0.13.15 ...
[info] Resolving org.scala-sbt.ivy#ivy;2.3.0-sbt-48dd0744422128446aee9ac31aa356ee203cc9f4 ...
[info] Resolving com.jcraft#jsch;0.1.50 ...
[info] Resolving org.scala-sbt#serialization_2.10;0.1.2 ...
[info] Resolving org.scala-lang.modules#scala-pickling_2.10;0.10.1 ...
[info] Resolving org.scalamacros#quasiquotes_2.10;2.0.1 ...
[info] Resolving org.json4s#json4s-core_2.10;3.2.10 ...
[info] Resolving org.json4s#json4s-ast_2.10;3.2.10 ...
[info] Resolving com.thoughtworks.paranamer#paranamer;2.6 ...
[info] Resolving org.spire-math#jawn-parser_2.10;0.6.0 ...
[info] Resolving org.spire-math#json4s-support_2.10;0.6.0 ...
[info] Resolving org.scala-sbt#run;0.13.15 ...
[info] Resolving org.scala-sbt#task-system;0.13.15 ...
[info] Resolving org.scala-sbt#tasks;0.13.15 ...
[info] Resolving org.scala-sbt#tracking;0.13.15 ...
[info] Resolving org.scala-sbt#cache;0.13.15 ...
[info] Resolving org.scala-sbt#testing;0.13.15 ...
[info] Resolving org.scala-sbt#test-agent;0.13.15 ...
[info] Resolving org.scala-sbt#test-interface;1.0 ...
[info] Resolving org.scala-sbt#main-settings;0.13.15 ...
[info] Resolving org.scala-sbt#apply-macro;0.13.15 ...
[info] Resolving org.scala-sbt#command;0.13.15 ...
[info] Resolving org.scala-sbt#template-resolver;0.1 ...
[info] Resolving org.scala-sbt#logic;0.13.15 ...
[info] Resolving org.scala-sbt#compiler-interface;0.13.15 ...
[info] Resolving org.scala-sbt#api;0.13.15 ...
[info] Resolving org.scala-sbt#compile;0.13.15 ...
[info] Resolving org.scala-sbt#persist;0.13.15 ...
[info] Resolving org.scala-sbt#process;0.13.15 ...
[info] Resolving org.scala-sbt#classpath;0.13.15 ...
[info] Resolving org.scala-sbt#relation;0.13.15 ...
[info] Resolving org.scalamacros#quasiquotes_2.10;2.0.1 ...
[info] Resolving org.scala-sbt#serialization_2.10;0.1.2 ...
[info] Resolving org.scala-lang.modules#scala-pickling_2.10;0.10.1 ...
[info] Resolving org.scala-sbt#test-agent;0.13.15 ...
[info] Resolving org.scala-sbt#logging;0.13.15 ...
[info] Resolving org.scala-sbt#command;0.13.15 ...
[info] Resolving org.scala-sbt#task-system;0.13.15 ...
[info] Resolving org.scala-sbt#compiler-integration;0.13.15 ...
[info] Resolving org.scala-lang#scala-reflect;2.10.6 ...
[info] Resolving org.scala-sbt#actions;0.13.15 ...
[info] Resolving org.scala-sbt#classfile;0.13.15 ...
[info] Resolving org.scala-sbt#ivy;0.13.15 ...
[info] Resolving org.scala-sbt#compiler-interface;0.13.15 ...
[info] Resolving org.scala-sbt#cache;0.13.15 ...
[info] Resolving org.scala-sbt#testing;0.13.15 ...
[info] Resolving org.scala-sbt#launcher-interface;1.0.1 ...
[info] Resolving org.json4s#json4s-core_2.10;3.2.10 ...
[info] Resolving org.scala-sbt#cross;0.13.15 ...
[info] Resolving org.scala-sbt#tasks;0.13.15 ...
[info] Resolving org.scala-sbt#apply-macro;0.13.15 ...
[info] Resolving org.scala-sbt#main;0.13.15 ...
[info] Resolving org.scala-sbt#collections;0.13.15 ...
[info] Resolving org.spire-math#jawn-parser_2.10;0.6.0 ...
[info] Resolving org.scala-sbt#io;0.13.15 ...
[info] Resolving org.scala-sbt#interface;0.13.15 ...
[info] Resolving org.json4s#json4s-ast_2.10;3.2.10 ...
[info] Resolving org.scala-lang#scala-library;2.10.6 ...
[info] Resolving org.scala-sbt#template-resolver;0.1 ...
[info] Resolving org.scala-tools.sbinary#sbinary_2.10;0.4.2 ...
[info] Resolving org.spire-math#json4s-support_2.10;0.6.0 ...
[info] Resolving org.scala-sbt#tracking;0.13.15 ...
[info] Resolving com.thoughtworks.paranamer#paranamer;2.6 ...
[info] Resolving org.scala-sbt#completion;0.13.15 ...
[info] Resolving org.scala-sbt#control;0.13.15 ...
[info] Resolving org.scala-sbt#sbt;0.13.15 ...
[info] Resolving org.scala-sbt#run;0.13.15 ...
[info] Resolving org.scala-sbt.ivy#ivy;2.3.0-sbt-48dd0744422128446aee9ac31aa356ee203cc9f4 ...
[info] Resolving org.scala-sbt#test-interface;1.0 ...
[info] Resolving com.jcraft#jsch;0.1.50 ...
[info] Resolving org.scala-lang#scala-compiler;2.10.6 ...
[info] Resolving jline#jline;2.14.3 ...
[info] Resolving org.scala-sbt#compiler-ivy-integration;0.13.15 ...
[info] Resolving org.scala-sbt#incremental-compiler;0.13.15 ...
[info] Resolving org.scala-sbt#logic;0.13.15 ...
[info] Resolving org.scala-sbt#main-settings;0.13.15 ...
[trace] Stack trace suppressed: run 'last *:update' for the full output.
[trace] Stack trace suppressed: run 'last *:ssExtractDependencies' for the full output.
[error] (*:update) Conflicting cross-version suffixes in: org.scala-lang.modules:scala-xml
[error] (*:ssExtractDependencies) Conflicting cross-version suffixes in: org.scala-lang.modules:scala-xml
[error] Total time: 1 s, completed 2017-5-18 1:02:48
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=384M; support was removed in 8.0

我的sbt配置文件

```
name := "spark"

version := "1.0"

scalaVersion := "2.12.2"

resolvers += "Cloudera Repository" at "https://repository.cloudera.com/content/repositories/releases/"
resolvers += "Spring Plugins Repository" at "http://repo.spring.io/plugins-release/"
resolvers += "Maven Repository" at "https://mvnrepository.com/"

libraryDependencies += "org.apache.camel" % "camel-test-spring" % "2.10.1" % "test" intransitive()

libraryDependencies += "com.google.guava" % "guava" % "16.0.1"
libraryDependencies += "commons-codec" % "commons-codec" % "1.10"
libraryDependencies += "commons-logging" % "commons-logging" % "1.2"
libraryDependencies += "org.codehaus.jackson" % "jackson-mapper-asl" % "1.9.13"
libraryDependencies += "jline" % "jline" % "2.12.1"

libraryDependencies += "org.apache.zookeeper" % "zookeeper" % "3.4.5-cdh5.7.1"

libraryDependencies += "org.apache.hadoop" % "hadoop-client" % "2.6.0-cdh5.7.1"
libraryDependencies += "org.apache.hadoop" % "hadoop-common" % "2.6.0-cdh5.7.1"
libraryDependencies += "org.apache.hadoop" % "hadoop-auth" % "2.6.0-cdh5.7.1"
libraryDependencies += "org.apache.hadoop" % "hadoop-annotations" % "2.6.0-cdh5.7.1"
libraryDependencies += "org.apache.hadoop" % "hadoop-yarn-server-nodemanager" % "2.6.0-cdh5.7.1"

libraryDependencies += "org.apache.hbase" % "hbase" % "1.2.0-cdh5.7.1"
libraryDependencies += "org.apache.hbase" % "hbase-client" % "1.2.0-cdh5.7.1"
libraryDependencies += "org.apache.hbase" % "hbase-common" % "1.2.0-cdh5.7.1"
libraryDependencies += "org.apache.hbase" % "hbase-server" % "1.2.0-cdh5.7.1"


libraryDependencies += "org.scala-lang" % "scala-compiler" % "2.12.2"
libraryDependencies += "com.typesafe.scala-logging" % "scala-logging_2.11" % "3.5.0" % "provided"

libraryDependencies += "org.mongodb" % "casbah_2.11" % "3.1.1"
libraryDependencies += "org.mongodb.spark" % "mongo-spark-connector_2.11" % "2.0.0"

libraryDependencies += "org.apache.spark" % "spark-core_2.11" % "2.1.0"
libraryDependencies += "org.apache.spark" % "spark-sql_2.11" % "2.1.0"
libraryDependencies += "org.apache.spark" % "spark-mllib_2.11" % "2.1.0"

出现error的地方字面理解是Scala版本冲突,即用了2.11版本的Scala,也用了2.12版本的Scala。但是到写这篇博客为止,去mvn中没有找到2.12版本的spark,所以放弃2.12.x版本的Scala,使用了2.11.8之后刷新sbt工程后,下载jar包正常也没有在报错。。

例如上面的
libraryDependencies += "org.apache.spark" % "spark-core_2.11" % "2.1.0"
其依赖的是
spark-core_2.11

总结一下:
所有jar包依赖的Scala版本号必须一致,不仅spark,还有mongodb等等。
以2.11结尾的jar(如spark-core_2.11)都依赖2.11.x版本的Scala,所以必须用2.11.x版本的Scala,否则会出现Scala包依赖冲突

修改后好用的sbt文件如下:

name := "spark"

version := "1.0"

scalaVersion := "2.11.8"

resolvers += "Cloudera Repository" at "https://repository.cloudera.com/content/repositories/releases/"
resolvers += "Spring Plugins Repository" at "http://repo.spring.io/plugins-release/"
resolvers += "Maven Repository" at "https://mvnrepository.com/"

//  所有jar包依赖的Scala版本号必须一致,
//  即所有以2.11结尾的jar(如spark-core_2.11)都依赖2.11.x版本的Scala,所以必须用2.11.x版本的Scala,否则会出现Scala包依赖冲突

libraryDependencies += "org.apache.camel" % "camel-test-spring" % "2.10.1" % "test" intransitive()

libraryDependencies += "com.google.guava" % "guava" % "16.0.1"
libraryDependencies += "commons-codec" % "commons-codec" % "1.10"
libraryDependencies += "commons-logging" % "commons-logging" % "1.2"
libraryDependencies += "org.codehaus.jackson" % "jackson-mapper-asl" % "1.9.13"
libraryDependencies += "jline" % "jline" % "2.12.1"

libraryDependencies += "org.apache.zookeeper" % "zookeeper" % "3.4.5-cdh5.7.1"

libraryDependencies += "org.apache.hadoop" % "hadoop-client" % "2.6.0-cdh5.7.1"
libraryDependencies += "org.apache.hadoop" % "hadoop-common" % "2.6.0-cdh5.7.1"
libraryDependencies += "org.apache.hadoop" % "hadoop-auth" % "2.6.0-cdh5.7.1"
libraryDependencies += "org.apache.hadoop" % "hadoop-annotations" % "2.6.0-cdh5.7.1"
libraryDependencies += "org.apache.hadoop" % "hadoop-yarn-server-nodemanager" % "2.6.0-cdh5.7.1"

libraryDependencies += "org.apache.hbase" % "hbase" % "1.2.0-cdh5.7.1"
libraryDependencies += "org.apache.hbase" % "hbase-client" % "1.2.0-cdh5.7.1"
libraryDependencies += "org.apache.hbase" % "hbase-common" % "1.2.0-cdh5.7.1"
libraryDependencies += "org.apache.hbase" % "hbase-server" % "1.2.0-cdh5.7.1"


libraryDependencies += "org.scala-lang" % "scala-compiler" % "2.11.8"
libraryDependencies += "com.typesafe.scala-logging" % "scala-logging_2.11" % "3.5.0" % "provided"

libraryDependencies += "org.mongodb" % "casbah_2.11" % "3.1.1"
libraryDependencies += "org.mongodb.spark" % "mongo-spark-connector_2.11" % "2.0.0"

libraryDependencies += "org.apache.spark" % "spark-core_2.11" % "2.1.0"
libraryDependencies += "org.apache.spark" % "spark-sql_2.11" % "2.1.0"
libraryDependencies += "org.apache.spark" % "spark-mllib_2.11" % "2.1.0"
  • 2
    点赞
  • 1
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值