CDH5.2.0 yarn启动失败

最新推荐文章于 2024-08-16 09:46:28 发布
最新推荐文章于 2024-08-16 09:46:28 发布
阅读量1.1k 收藏
点赞数
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/wenwen1709/article/details/42395877
版权


STARTUP_MSG:   build = http://github.com/cloudera/hadoop -r e1f20a08bde76a33b79df026d00a0c91b2298387; compiled by 'jenkins' on 2014-10-11T21:00Z
STARTUP_MSG:   java = 1.7.0_67
************************************************************/
2015-01-04 17:46:19,477 INFO org.apache.hadoop.yarn.server.nodemanager.NodeManager: registered UNIX signal handlers for [TERM, HUP, INT]
2015-01-04 17:46:20,230 INFO org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService: Using state database at /tmp/hadoop-yarn/yarn-nm-rn-nm-state for recovery
2015-01-04 17:46:20,257 INFO org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService$LeveldbLogger: Recovering log #323
2015-01-04 17:46:20,259 INFO org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService$LeveldbLogger: Delete type=3 #322

2015-01-04 17:46:20,259 INFO org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService$LeveldbLogger: Delete type=0 #323

2015-01-04 17:46:20,278 INFO org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService: Loaded NM state version info 1.0
2015-01-04 17:46:20,562 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.contanerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher
2015-01-04 17:46:20,563 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.appliicationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ApplicationEventDispatcher
2015-01-04 17:46:20,564 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localLocalizationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService
2015-01-04 17:46:20,564 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxSeType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices
2015-01-04 17:46:20,565 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitrsMonitorEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl
2015-01-04 17:46:20,565 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcersLauncherEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher
2015-01-04 17:46:20,575 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.ContainerManagerEventTss org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl
2015-01-04 17:46:20,576 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.NodeManagerEventType fg.apache.hadoop.yarn.server.nodemanager.NodeManager
2015-01-04 17:46:20,605 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2015-01-04 17:46:20,657 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2015-01-04 17:46:20,657 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NodeManager metrics system started
2015-01-04 17:46:20,675 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.logha.LogHandlerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService
2015-01-04 17:46:20,676 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: per directory file limit = 819
2015-01-04 17:46:20,691 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localLocalizerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker
2015-01-04 17:46:20,707 WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: The Auxilurary Service named 'mapreduce_shuffle' in the on is for class class org.apache.hadoop.mapred.ShuffleHandler which has a name of 'httpshuffle'. Because these are not the same tools trying to send ServiceDd Service Meta Data may have issues unless the refer to the name in the config.
2015-01-04 17:46:20,707 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Adding auxiliary service httpshuffle, "mapreduce_shuffle
2015-01-04 17:46:20,788 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl:  Using ResourceCalculatorPlugin : org.op.yarn.util.LinuxResourceCalculatorPlugin@43a54967
2015-01-04 17:46:20,788 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl:  Using ResourceCalculatorProcessTree :
2015-01-04 17:46:20,789 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Physical memory check enabled: true
2015-01-04 17:46:20,789 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Virtual memory check enabled: false
2015-01-04 17:46:20,867 INFO org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImn state INITED; cause: java.lang.NullPointerException
java.lang.NullPointerException
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.recoverContainer(ContainerManagerImpl.java:289)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.recover(ContainerManagerImpl.java:252)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:235)
        at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
        at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
        at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:250)
        at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
        at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:445)
        at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:492)
2015-01-04 17:46:20,876 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService: org.apache.hadoop.yarn.server.ncontainermanager.logaggregation.LogAggregationService waiting for pending aggregation during exit
2015-01-04 17:46:20,877 INFO org.apache.hadoop.service.AbstractService: Service NodeManager failed in state INITED; cause: java.lang.NullPointerException
java.lang.NullPointerException
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.recoverContainer(ContainerManagerImpl.java:289)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.recover(ContainerManagerImpl.java:252)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:235)
        at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
        at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
        at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:250)
        at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
        at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:445)
        at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:492)
2015-01-04 17:46:20,878 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping NodeManager metrics system...
2015-01-04 17:46:20,878 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NodeManager metrics system stopped.
2015-01-04 17:46:20,878 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NodeManager metrics system shutdown complete.
2015-01-04 17:46:20,878 FATAL org.apache.hadoop.yarn.server.nodemanager.NodeManager: Error starting NodeManager
java.lang.NullPointerException
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.recoverContainer(ContainerManagerImpl.java:289)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.recover(ContainerManagerImpl.java:252)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:235)
        at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
        at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
        at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:250)
        at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
        at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNod
        at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManage
2015-01-04 17:46:20,881 INFO org.apache.hadoop.yarn.server.nodemanager.NodeManag

 

 

 

 

解决方法：rm -rf /tmp/hadoop-yarn

wenwen1709
关注
  • 0
    点赞
  • 踩
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
搭建CDH 阿里云 (Step 2: 启动yarn)
xw514124202的博客
08-04 727
Yarn HA 文档 Yarn HA 文档 配置 Yarn 分配集群资源 文档地址 yarn-site.xml 配置 <!-- Base Configs --> <property> <description>Classpath for typical applications.</description&gt
CDH集群YARN动态资源池的规划与管理.docx
10-11
YARN动态资源池的规划与管理是优化集群性能的关键环节，尤其在CDH（Cloudera Distribution Including Apache Hadoop）这样的企业级Hadoop平台中更为重要。 YARN提供了三种调度器：FIFO Scheduler、Capacity ...
cdh启动yarn失败
yt438936731的博客
05-21 1524
如下，运用cloudera搭建集群启动yarn失败。 1、查找 cloudera-scm-server与cloudera-scm-agent日志文件 排错 tail -f /opt/cloudera-manager/cm-5.14.1/log/cloudera-scm-server/cloudera-scm-server.log tail -f /opt/cloudera-manager/cm-5.14.1/log/cloudera-scm-agent/cloudera-scm-agent.
yarn启动失败原因
Java大数据
11-03 9051
今天在做mapReduce 的学习开发时候，当我用start-yarn.sh启动yarn集群的时候一直都是启动失败，上网查了很久都说是yarn-site.xml 问题，自己核对了好几遍都没有找到问题，最后不得已一个一个核对，才发现问题，所以，就必须得记录一下，这一个巨大的坑。我的配置如下： 在这里插入代码片 <property> <name>yarn.resourcema...
yarn node manager启动失败
huguoping830623的专栏
07-14 2004
/data/lib/hadoop-yarn mv yarn-nm-recovery /tmp/20160225bak       Application application_1456459092764_0002 failed 2 times due to AM Container for appattempt_1456459092764_0002_000002 e
解决yarn start启动失败的问题
weixin_43886986的博客
03-05 4492
这里我使用了一个气泡按钮，确定的话就会跳转到支付页面 现在来解决按钮跳转的问题 function confirm(props) { //console.log(props.target.ownerDocument.location.pathname); props.target.ownerDocument.location.pathname = "/home" //this.props.history.push("/home") //props.history.replace("/home"
CDH+切换用户失败+解决方案
03-01
- 在完成以上步骤后，建议重新启动CDH集群服务，确保所有配置更改生效。 - 如果修改了多个用户的登录Shell，请确保这些用户确实需要交互式登录权限，并注意可能的安全风险。 #### 四、扩展知识 - **关于`/sbin/...
CDH+集群安全启动和关闭.doc
03-11
CDH集成了众多开源组件，如HDFS、YARN、MapReduce、Hive等，为用户提供了一套完整的处理海量数据的技术解决方案。 #### 二、集群安全启动与关闭的重要性 在企业环境中，为了保障业务连续性和数据安全性，在进行...
native(hadoop-2.5.0-cdh5.2.0).tar
10-19
标题 "native(hadoop-2.5.0-cdh5.2.0).tar" 提供的是一个针对Hadoop 2.5.0 CDH5.2.0版本的本地库文件集合，它已经过编译，可以用于在特定操作系统上支持Hadoop运行。CDH（Cloudera Distribution Including Apache ...
cdh6.3.1 FLINK_ON_YARN-1.14.0.jar
02-14
CDH6.3.1集成安装flink on yarn服务（通过parcel制作生成的csd文件） 文件名：FLINK_ON_YARN-1.14.0.jar 相关信息如下： 1. flink版本号：1.14.0 2. 系统版本：CentOS7.7 3. CDH版本：6.3.1 4. 扩展版本号：BIN-...
Hadoop启动yarn报错
NullNullAgo的博客
01-06 1690
Hadoop启动yarn的时候，无法获取"/opt/module/hadoop-3.1.1/logs/hadoop-ajie-nodemanager-hadoop102.out" 的文件状态(stat): 没有那个文件或目录 解决办法 进入 vim /opt/module/hadoop-3.1.3/etc/hadoop/workers，删除localhost，如果已经删除，就是hostname可能写的有问题，hostname前后不允许有空格 ...
【解决】hive on yarn 任务启动失败：JDWP Transport dt_socket failed to initialize，TRANSPORT_INIT(510)
hiliang521的博客
11-22 643
【解决】hive on yarn 任务启动失败：JDWP Transport dt_socket failed to initialize，TRANSPORT_INIT(510)
CDH 6.3.2启用YARN高可用
Aidon博客
02-28 513
CDH 6.3.2启用YARN高可用，生产中YARN通常都是高可用，故生产环境下几乎都是需要将CDH中的YARN高可用开启。
flink on yarn集成到CDH 启动几分钟后自动挂掉
最新发布
weixin_36047538的博客
08-16 12
我整理的一些关于【CD,HA】的项目学习资料（附讲解～～）和大家一起分享、学习一下：https://d.51cto.com/xltfovFlink on YARN 集成到 CDH 启动几分钟后自动挂掉的解析 Apache Flink 是一个分布式流处理框架，能够帮助我们处理实时数据流。Flink on YARN 是将 F...
【无标题】搭建yarn自动故障时，报错解决方法
Shenshen2017的博客
01-04 1273
报错信息：Service ResourceManager failed in state INITED org.apache.hadoop.HadoopIllegalArgumentException: The HA Configuration has multiple addresses that match local node's address. 配置的三台机器，只有一台resourcemanager和nodemanager启来了，其余两台的resourcemanager没启动，nodemanag
hadoop报错——ResourceManager无法启动
weixin_63507910的博客
04-14 2614
本次的遇到的问题为启动Hadoop集群后，jps查看进程中，ResourceManage节点没有启动，导致无法访问http://localhost:8088，通过查看ResourceManage的启动日志，日志中给出的错误为，笔者初步判断为yarn-site.xml文件未填写Zookeeper的端口，以及设置开启自动恢复和故障自动转移。
RM源码之org.apache.hadoop.service.AbstractService分析
weixin_30800807的博客
08-19 380
org.apache.hadoop.service.AbstractService父类为org.apache.hadoop.service.Service 其中Service主要变量和方法为： 状态改变枚举 /** Constructed but not initialized */ NOTINITED(0, "NOTINITED"), /** Initialized but...
hadoop集群启动yarn时出现的问题和解决方法
qianyulin
06-01 5727
1、问题 使用start-all.sh 启动hdfs和yarn集群 ，启动后发现resourcemanager没启动，查看日志报错： 2020-06-01 09:42:21,280 INFO org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.yarn.server.resourcemanager.AdminService failed in state INITED; cause: org.apache.had
cdh flink on yarn
08-17
根据引用中的信息，CDH5.16.2上配置Flink on yarn可以通过flink官网提供的集成方式和HA配置方式来进行。具体的配置方式可以参考官方文档来完成。此外，根据引用中的信息，使用flink的web ui可以重新访问flink on ...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值