hive一直卡住不动，jps时ResourceManager 没启动

最新推荐文章于 2022-06-19 21:20:45 发布

wangxj_2017

最新推荐文章于 2022-06-19 21:20:45 发布

阅读量1.1w

点赞数 7

分类专栏： hive

本文链接：https://blog.csdn.net/wangxj_2017/article/details/78148551

版权

hive 专栏收录该内容

3 篇文章 0 订阅

订阅专栏

在使用hive时，遇到只要查询涉及到跑MR的时候，就会卡住不动；

例如

hive> select id from db_hive.student;
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator

就会卡住不动。这里一般要jps 看看ResourceManager 是否启动了。

于是看log ,发现有异常：

2017-10-01 11:01:06,831 INFO org.apache.hadoop.service.AbstractService: Service RMActiveServices failed in state STARTED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.n
et.BindException: Problem binding to [node3:8031] java.net.BindException: Cannot assign requested address; For more details see:  http://wiki.apache.org/hadoop/BindException
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.net.BindException: Problem binding to [node3:8031] java.net.BindException: Cannot assign requested address; For more details see: 
 http://wiki.apache.org/hadoop/BindException
	at org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl.getServer(RpcServerFactoryPBImpl.java:139)
	at org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC.getServer(HadoopYarnProtoRPC.java:65)
	at org.apache.hadoop.yarn.ipc.YarnRPC.getServer(YarnRPC.java:54)
	at org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService.serviceStart(ResourceTrackerService.java:163)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
	at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceStart(ResourceManager.java:511)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startActiveServices(ResourceManager.java:922)
	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:967)
	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:963)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642)
	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:963)
	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1009)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1143)
Caused by: java.net.BindException: Problem binding to [node3:8031] java.net.BindException: Cannot assign requested address; For more details see:  http://wiki.apache.org/hadoop/BindException
	at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:719)
	at org.apache.hadoop.ipc.Server.bind(Server.java:419)
	at org.apache.hadoop.ipc.Server$Listener.<init>(Server.java:561)
	at org.apache.hadoop.ipc.Server.<init>(Server.java:2166)
	at org.apache.hadoop.ipc.RPC$Server.<init>(RPC.java:995)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server.<init>(ProtobufRpcEngine.java:505)
	at org.apache.hadoop.ipc.ProtobufRpcEngine.getServer(ProtobufRpcEngine.java:480)
	at org.apache.hadoop.ipc.RPC$Builder.build(RPC.java:840)
	at org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl.createServer(RpcServerFactoryPBImpl.java:169)
	at org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl.getServer(RpcServerFactoryPBImpl.java:132)
	... 17 more

说 node3 结点绑定异常；这是什么鬼，不知道呀。

于是网上说：说是Namenode和ResourceManger如果不是同一台机器，不能在NameNode上启动 yarn，应该在ResouceManager所在的机器上启动yarn。

然后去看配置文件 yarn-site.xml

<property>
<name>yarn.resourcemanager.hostname</name>
<value>node3</value>
</property>

似乎找问题了：我是在node1 运行start-dfs.sh 后再运行 start-yarn.sh ,但是jps 时，没有发现node1 上的ResourceManager进程。我的集群中NameNode和ResourceManager不在同一台机器上。

于是修改 yarn-site.xml :

<property>
<name>yarn.resourcemanager.hostname</name>
<value>node1</value>
</property>

之后重新启动集群后，运行hive，就可以查询：select id from db_hive.student; 这种要跑MR的查询语句了。

总结：

1. namenode和ResourceManager 在同一台机器上时，启动start-yarn.sh 后就没问题。

2 .namenode和ResourceManager 不在同一台机器上时，要在ResouceManager所在的机器上启动。（这个没试过）