报错: Only one SparkContext may be running in this JVM (see SPARK-2243).

最新推荐文章于 2024-04-10 22:30:00 发布

懒散样子

最新推荐文章于 2024-04-10 22:30:00 发布

阅读量5.6k

点赞数 1

分类专栏：技术 API 文章标签： spark streaming kafka

本文链接：https://blog.csdn.net/weixin_44175698/article/details/86645518

版权

技术同时被 2 个专栏收录

27 篇文章 0 订阅

订阅专栏

API

1 篇文章 0 订阅

订阅专栏

org.apache.spark.SparkException: Only one SparkContext may be running in this JVM (see SPARK-2243). To ignore this error, set spark.driver.allowMultipleContexts = true. The currently running SparkContext was created at:
org.apache.spark.api.java.JavaSparkContext
出现这个问题的原因就是你创建了多个sparkcontext,就像下面这种用法,只需要干掉JavaSparkContext就可:
SparkConf conf = new SparkConf()
.setAppName(“myapplication”).setMaster(“local[4]”);
JavaSparkContext jsc = new JavaSparkContext(conf);
JavaStreamingContext stream = new JavaStreamingContext(conf , Durations.seconds(10));

解决这个问题两种方式:
方式1：
SparkConf conf = new SparkConf()
.setAppName(“myapplication”) .setMaster(“local[4]”);
JavaStreamingContext stream = new JavaStreamingContext(conf, Durations.seconds(10));
方式2：
SparkConf conf = new SparkConf()
.setAppName(“myapplication”).setMaster(“local[4]”);
JavaSparkContext jsc = new JavaSparkContext(conf);
JavaStreamingContext stream = new JavaStreamingContext(jsc , Durations.seconds(10));