搭建运行spark streaming过程遇到的问题
问题描述:
完成spark streaming之后执行mvn package之后出现如下错误:
Failed to execute goal org.scala-tools:maven-scala-plugin:2.15.2:testCompile (default) on project ProjectXXX
出现问题原因:
这个问题,是IDEA里的Scala版本 与 本地win里的Scala版本不合理
解决方法:
(1)下载对应的本地scala版本(我的pom.xml中的是scala2.11.12)到windows环境中,我下载的是scala2.11.12.mmsi版本,下载完成安装之后,默认其已经配置好了系统环境变量path;
执行scala -version
命令可以看到如下表示成功:
C:\Users\user>scala -version
Scala code runner version 2.11.12 -- Copyright 2002-2017, LAMP/EPFL
(2)然后在idea的已有项目名称上右键选择“open muddle settings”->“Global Libraries”,点击加号添加相应的scala sdk版本,确定即可!
最后好从新执行相应的mvn package命令打包成功!!
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ MySparkStreamTest1 ---
[INFO] Building jar: F:\bigdata\IDEA-workspace\MySparkStreamTest1\target\MySparkStreamTest1-1.0-SNAPSHOT.jar
[INFO]
[INFO] --- maven-assembly-plugin:3.0.0:single (make-assembly) @ MySparkStreamTest1 ---
[INFO] Building jar: F:\bigdata\IDEA-workspace\MySparkStreamTest1\target\MySparkStreamTest1-1.0-SNAPSHOT-jar-with-dependencies.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:44 min
[INFO] Finished at: 2019-06-28T10:59:59+08:00
[INFO] ------------------------------------------------------------------------