spark--IntelliJIdea中jar包提交到spark报错解决

最新推荐文章于 2021-05-22 17:43:58 发布

prettysky123

最新推荐文章于 2021-05-22 17:43:58 发布

阅读量874

点赞数

分类专栏： spark 文章标签： IntelliJ Maven Spark

本文链接：https://blog.csdn.net/prettysky123/article/details/79177290

版权

spark 专栏收录该内容

3 篇文章 0 订阅

订阅专栏

在IntelliJ上利用Maven开发spark程序，今天遇到一个问题：在IntelliJ上可以运行的程序，利用Bulid Artifacts来打好jar包之后，再用spark-submit 提交运行之后老是报错：

$ spark-submit --class MySpark --master local sparktest.jar 
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/spark/jars/slf4j-log4j12-1.7.16.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
java.lang.ClassNotFoundException: MySpark
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:348)
    at org.apache.spark.util.Utils$.classForName(Utils.scala:230)
    at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:732)
    at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180)
    at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205)
    at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:119)
    at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)