Spark提交任务Initial job has not accepted any resources; check your cluster UI to ensure that workers ar

最新推荐文章于 2023-05-25 10:04:56 发布
最新推荐文章于 2023-05-25 10:04:56 发布
阅读量1.1w 收藏 2
点赞数
分类专栏： Spark
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/u012848709/article/details/85425249
版权

楔子

spark集群提交任务

Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources

问题

虚拟机内存 和内核不够大,需要调整
本人是16G 的笔记本
调整后如图

在这里插入图片描述

虚拟机配置

两个work 节点配置如下图

在这里插入图片描述

后记

简单粗暴 加大内存。仅作为学习的话，使用eclipse 运行本地也是可以的

千里草竹
关注
专栏目录
spark-summit 中出现Initial job has not accepted any resources; check your cluster UI to ensure that wor
简单就好
03-23 3274
spark-summit 中出现Initial job has not accepted any resources; check your cluster UI to ensure that…Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered
spark报错Initial job has not accepted any resources； check your cluster UI to ensure that workers are
qq_39954916的博客
10-02 1610
报错: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 我在jupyter notebook中以： spark = SparkSession.builder.master('spark://hadoop101:7077').appName('apply123').getOrCre..
erImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are
图特摩斯科技-博客
08-27 1344
erImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
spark报错：WARN TaskSchedulerImpl: Initial job has not accepted any resources； check your cluster UI...
Iv_zzy的博客
11-05 7514
Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure
羽化成蝶
06-06 1378
1：遇到这个问题是在启动bin/spark-shell以后，然后呢，执行spark实现wordcount的例子的时候出现错误了，如： scala> sc.textFile("hdfs://slaver1:9000/hello.txt").flatMap(_.split(" ")).map((_,1)).reduceByKey(_ + _).collect 执行上面操作之前我是这样启动我...
Initial job has not accepted any resources
浮世与人心的博客
04-28 523
关于spark ：Initial job has not accepted any resources 早几天，在客户端提交job的时候 突然遇到了一个新的问题，代码没变，环境没变，出现了这个异常（因为问题已经解决了，就不贴图了），感觉很莫名奇妙。 在集群的日志中 ，我们可以看到这个这样一个错误： 在网上 找了很久的资料，说可能是 ip不对，去服务器，特意去ping了一下，能ping通，就给排掉...
记录-----解决Spark之submit任务时的Initial job has not accepted any resources; check your cluster UI to ensu问题
gk91620的博客
09-12 2375
大多数是集群资源有限导致的问题。注意合理分配资源   原则1：首先，保证至少有一个executor可以成功启动，否则，提交的spark应用是无法跑起来的 如何保证？ 第一点：--executor-cores <= SPARK_WORKER_CORES(spark-env.sh中设置的参数)，否则会出现资源不足报错   第二点：--executor-memorry <= SP...
Spark运行时遇到的错误 scheduler.TaskSchedulerImpl: Initial job has not accepted any resources
01-07
scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your clus ter UI to ensure that workers are registered and have sufficient resources 调度器：初始化Job时没有足够的资源，...
WARN TaskSchedulerImpl: Initial job has not accepted any resources； check your……解决方法，两种可能原因
weixin_48620718的博客
04-14 8669
WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources，意思是：初始作业未接受任何资源；请检查群集UI以确保工作进程已注册并且有足够的资源。 出错可能有两种原因。 第一种：是spark节点的内存满了，内存可以在spark的配置文件—>spark-
Initial job has not accepted any resources; check your cluster UI to ensure that workers are registe
热门推荐
InforStack
01-11 1万+
在启动start-master.sh的时候，还需要启动一个start-slave.sh;
Spark报错~WARN TaskSchedulerImpl: Initial job has not accepted any resources； check your clu
最新发布
qq_61324603的博客
05-25 541
内存不知道满不满可以去独立集群Web界面查看应用的运行情况，打开浏览器，访问master:8080,节点的内存满了，内存可以在spark的配置文件—>spark-env.sh中可以看到。意思是：初始作业未接受任何资源；请检查群集UI以确保工作进程已注册并且有足够的资源。查看各个节点内存和使用情况，可以看到我的三个节点的内存有使用所以是1的。
Initial job has not accepted any resources； check your cluster UI to ensure that workers are registe
qq_14945847的博客
05-07 791
Initial job has not accepted any resources; check your cluster UI to ensure that workers are registe
spark - Initial job has not accepted any resources...没有接受到数据
临地的博客
08-05 877
1.表现与原因 表现为将spark streaming提交到集群时（setMaster()），没有接受到任何数据，并打印 Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 原因为集群没有资源执行job，基本是因为只启动了msater，没有启动work 2.解决方法 启动work即
Spark-shell执行RDD操作时报Initial job has not accepted any resources问题解决过程
weixin_44042314的博客
03-05 999
因为资源紧缺，用一台16G内存的虚拟机搭建了整套CDH6.1的大数据环境，全部服务启动后仅剩不到1G的内存了。 测试spark-shell时，在执行RDD的action类操作时总是报Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and h...
Initial job has not accepted any resources; check your cluster UI to ensure that workers are...
Mr Zhu's little cute
08-06 1593
在集群上运行spark应用程序时，出现： Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 如下图所示：原因是下图最后一段代码：pairRDD.foreach(println) 应该...
Spark报错：Initial job has not accepted any resources
BigData_Mining的博客
06-26 1597
警告信息：初始化job的时候，没有获取到任何资源，检查你的集群，确保workers已经被注册和有足够的内存资源。问题出现的可能原因：worker节点没有启动；你分配的内存资源超过worker节点的内存容量（就是内存溢出了）...
spark提示Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
05-14
这个错误信息通常表示你的Spark应用程序没有获得足够的资源来启动。这可能是由于以下原因之一导致的： 1. 集群中没有足够的可用资源来满足您的应用程序的要求。您可以检查您的集群管理器（例如YARN或Mesos）的UI，...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值