提交yarn任务报错空指针org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster

新建的hadoop集群提交yarn任务后报错,hive任务也执行不了。
报错如下:

ERROR [Listener at 0.0.0.0/45871] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.NullPointerException
    at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.register(RMCommunicator.java:178)
    at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.serviceStart(RMCommunicator.java:122)
    at org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.serviceStart(RMContainerAllocator.java:280)
    at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter.serviceStart(MRAppMaster.java:979)
    at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
    at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:121)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1293)
    at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$6.run(MRAppMaster.java:1761)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1757)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1691)
Caused by: java.lang.NullPointerException
    at org.apache.hadoop.mapreduce.v2.app.client.MRClientService.getHttpPort(MRClientService.java:177)
    at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.register(RMCommunicator.java:159)
    ... 14 more

看了下这个空指针MRClientService.getHttpPort(MRClientService.java:177)
@Override
public int getHttpPort() {
return webApp.port();
}
webApp的port没有获取到, 再看port()方法就是获取httpServer地址的端口
public int port() {
InetSocketAddress addr = checkNotNull(httpServer, “httpServer”)
.getConnectorAddress(0);
return addr == null ? -1 : addr.getPort();
}

看网上说是yarn.resourcemanager.webapp.address没配置导致没能获取到port。
在yarn-site.xml中配置了yarn.resourcemanager.webapp.address之后就正常了。

如果是HA机制下yarn-site.xml需要加入以下配置:

<!-- 指定 rm1 的主机名 -->
<property>
	<name>yarn.resourcemanager.hostname.rm1</name>
	<value>xxx1</value>
</property>
<!-- 指定 rm1 的 web 端地址 -->
<property>
<name>yarn.resourcemanager.webapp.address.rm1</name>
<value>xxx1:8088</value>
</property>

<!-- 指定 rm2 的主机名 -->
<property>
	<name>yarn.resourcemanager.hostname.rm2</name>
	<value>hadoop103</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.address.rm2</name>
<value>xxx2:8088</value>
</property>

可以参考这个配置https://blog.csdn.net/m0_51111980/article/details/126085577
xxx1主机名的值从yarn.resourcemanager.hostname中找,如果是HA那么是类似这样的

参考:
[1] https://blog.csdn.net/weixin_44812855/article/details/112716670
[2] https://blog.csdn.net/m0_51111980/article/details/126085577
[3] https://www.cnblogs.com/dreamboy/archive/2018/07/10/9289631.html
[4] https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/ResourceManagerHA.html

  • 0
    点赞
  • 1
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值