spark on yarn，Application application failed 2 times due to AM Container for app解决方法

最新推荐文章于 2025-01-12 14:57:20 发布
原创 最新推荐文章于 2025-01-12 14:57:20 发布 · 2w 阅读 · 3 ·
CC 4.0 BY-SA版权
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。

本文详细记录了在处理Applicationapplication失败问题时，发现AMContainer因exitCode:1而退出的具体情况。通过检查日志和应用跟踪页面，最终定位到类加载问题，并提供了解决方案，即添加所需的jar包。

client token: N/A
     diagnostics: Application application_1543827900491_0008 failed 2 times due to AM Container for appattempt_1543827900491_0008_000002 exited with  exitCode: 1
For more detailed output, check application tracking page:http://cdh6:8088/proxy/application_1543827900491_0008/Then, click on links to logs of each attempt.
 

 去http://cdh6:8088/proxy/application_1543827900491_0008/Then 查看日志 如下图

点进去后发现具体原因如图

发现是类加载不到，添加jar包即可

【spark】due to AM Container for x exited with exitCode: -1000.which is below the minimum allowed 1000
SHOKO2022
05-03 1463
spark操作kudu,提交spark程序报错解决：Requested user spark is not whitelisted and has id 978,which is below the minimum allowed 1000
PYSPARK ON YARN报错集合
weixin_45621200的博客
08-05 3526
pyspark报错整理
TezSession has already shutdown.Application XXX failed 2 times due to AM Container
weixin_43923463的博客
03-26 2248
错误信息 org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown. Application application_1648284477641_0003 failed 2 times due to AM Container for appattempt_1648284477641_0003_000002 exited with exitCode: 1 报错原因 hadoop集群上从机上运行的Container试
Flink 任务提交失败 Could not deploy Yarn job cluster
Baron_ND的博客
09-08 1万+
1.Flink提价任务，读取Hbase数据和hdfs数据，需要单独倒入一些hbase的依赖jar，所以加入了maven依赖，本地能正常测试运行，但是发布到yarn上运行报错，所以测试读取hdfs文件的时候，去掉依赖就可以了。 2.加入打包依赖的时候，注意加入过滤不需要的文件，不然会报错。 org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: Could not depl.
Spark集群环境搭建( Spark On YARN模式)
天涯芳草
04-08 2371
Spark On YARN模式的搭建比较简单,仅仅需要在YARN集群的”一个节点”上安装Spark即可, 该节点可作为提交Spark应用程序到YARN集群的客户端。Spark本身的Master节点和Worker节点不需要启动,Spark On Yarn集群的部署不依赖Standalone集群。 一：下载scala安装包 下载地址如下： https://www.scala-lang.org/d...
Application application_xxx failed 2 times due to AM Container for attempt_xxx exited withexitCode:
m0_53867640的博客
08-01 5161
通过上面的两张图片不难看出，主要是存在两个问题。结果显示块状态是健康的，即并没有块损坏这一说法。
Application application xxx failed 2 times due to AM Container for appattempt_xxx exited with exit
weixin_44366140的博客
02-06 1万+
21/02/06 10:06:37 INFO mapreduce.Job: map 0% reduce 0% 21/02/06 10:06:37 INFO mapreduce.Job: Job job_1612576553072_0003 failed with state FAILED due to: Application application_1612576553072_0003 failed 2 times due to AM Container for appattempt_161257655
spark 在集群运行时遇到的一些问题_am container for appattempt exitcode 13
2401_89829398的博客
01-12 491
参考文献： https://stackoverflow.com/questions/15571584/writing-to-hdfs-could-only-be-replicated-to-0-nodes-instead-of-minreplication。这个问题的原因在于RDD发算子中不能再操作其他的RDD，所以一定要使用其他RDD中的数据，可以考虑将其他RDD转成别的类型，再进行广播，比如下面的代码就是讲JavaRDD转成List。a, 关闭hadoop目前的所有进程 stop-all.sh。
Application Application_174168431691_13846 failed 2 times (globalimit -5 localimit is =2) due to AM Container for appattempt_1741684431691_13846_00002 exted with exitCode:1 Failing this attemptDiagnostics: [2025-03-20 15:51:46,920]Exception from container-launch
03-21
嗯，用户遇到了YARN应用失败的问题，AM容器的退出代码是1，诊断信息显示container-launch异常。首先，我需要根据提供的引用内容来分析可能的原因。根据引用[1]和[4]，退出代码1通常意味着容器启动时遇到了非零退出，...
Exception in thread "main" org.apache.spark.SparkException: Application application_1745535390776_3862 failed 2 times due to AM Container for appattempt_1745535390776_3862_000002 exited with exitCode: 1 Failing this attempt.Diagnostics: [2025-04-30 15:49:46.496]Exception from container-launch. Container id: container_e166_1745535390776_3862_02_000001 Exit code: 1 [2025-04-30 15:49:46.496]Container exited with a non-zero exit code 1. Error file: prelaunch.err. Last 4096 bytes of prelaunch.err : Last 4096 bytes of stderr : Error: Could not find or load main class org.apache.spark.deploy.yarn.ExecutorLauncher [2025-04-30 15:49:46.496]Container exited with a non-zero exit code 1. Error file: prelaunch.err. Last 4096 bytes of prelaunch.err : Last 4096 bytes of stderr : Error: Could not find or load main class org.apache.spark.deploy.yarn.ExecutorLauncher For more detailed output, check the application tracking page: http://bdpsit01:8088/cluster/app/application_1745535390776_3862 Then
最新发布
05-01
好的，我现在要解决用户遇到的Spark应用在YARN上运行时出现的错误：Could not find or load main class org.apache.spark.deploy.yarn.ExecutorLauncher。这个问题看起来和类路径配置有关，特别是ExecutorLauncher类...
解决 Application xxx failed 2 times due to AM Container for xxx exited with exitCode: 13 问题
qq_37354060的博客
03-21 1万+
出现问题要首先检查hostname是不是与hadoop、spark配置的一致，不要想当然的认为是一样的，先检查一下最基础的，如果还是不能解决，就要在网上搜索一下了，通常情况下，问题产生的原因都是因为忽略了某个地方的小细节。
异常：Application failed 2 times due to AM Container
热门推荐
Thinking
11-27 2万+
文章来源：http://ju.outofmemory.cn/entry/74294 异常：Application failed 2 times due to AM Container 极豆技术博客 » 极豆技术博客 2014-04-28 3248 阅读 hadoop 安装完Hadoop2版本后，启动测试程序报如下异常： /04/28 15:29:16 INFO
spark on yarn：due to AM Container for appattempt_xx_xx exited with exitCode:-1000
weixin_44455388的博客
04-28 3719
新部署的hadoop和spark集群，使用spark on yarn模式执行一个简单的spark程序包，spark程序的代码逻辑是读取oracle数据库，读取后对数据做简单处理，然后保存。比较匪夷所思的是，spark代码逻辑执行成功，经过处理的数据成功保存到数据库，但是yarn页面显示spark程序报错。 报错信息如下： 按这篇文章（https://www.e-learn.cn/content/wangluowenzhang/874518）的解决方法，无法解决。 网上找了很多都没有找到解决方。 由于是新部
Hadoop之——Hadoop3.x运行自带的WordCount报错Container exited with a non-zero exit code 1.
冰河的专栏
06-26 6555
转载请注明出处：https://blog.csdn.net/l1028386804/article/details/93750832 问题： 今天，基于Hadoop3.2.0搭建了Hadoop集群，对NameNode和Yarn做了HA，但是在运行Hadoop自带的WordCount程序时报错了，具体报错信息为： 2019-06-26 16:08:50,513 INFO mapreduce....
spark 问题
CottonDuke的博客
10-15 1530
Q:Application application_1505791560225_0911 failed 2 times due to AM Container for appattempt_1505791560225_0911_000002 exited with exitCode: -1000 For more detailed output, check application trac
spark运行任务报错：Container [...] is running beyond physical memory limits. Current usage: 3.0 GB of 3 GB ...
aitiejue5146的博客
07-10 1286
spark版本：1.6.0 scala版本：2.10 报错日志： Application application_1562341921664_2123 failed 2 times due to AM Container for appattempt_1562341921664_2123_000002 exited with exitCode: -104 For more d...
执行hadoop-3.3.0 wc 报错Container exited with a non-zero exit code 1. Error file: prelaunch.err.
top_explore的博客
01-27 8454
报错 Application application_1611732673035_0001 failed 2 times due to AM Container for appattempt_1611732673035_0001_000002 exited with exitCode: 1 Failing this attempt.Diagnostics: [2021-01-27 02:32:14.002]Exception from container-launch. Container id:
Application application_id failed 2 times due to ApplicationMaster for attempt xx timed out解决办法
berkely心路的博客
05-25 6961
spark-submit 提交任务失败，报错详情： Application application_1609129417468_1602993 failed 2 times due to ApplicationMaster for attempt appattempt_1609129417468_1602993_000002 timed out. Failing the application. 排查为ApplicationMaster 为子任务尝试次数超时，于是尝试添加以下参数： --conf s.
hue访问spark编辑器，有时创建任务会失败
qq_29891169的博客
10-12 1444
环境：hdp2.6.x，ambari2.5.x 报错内容如下： 无法在群集中创建 Spark 会话：YARN Diagnostics: Application application_1632379940636_0057 failed 1 times (global limit =2; local limit is =1) due to AM Container for appattempt_1632379940636_0057_000001 exited with exitCode: -1000 ..
评论 2
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值