hadoop 0.0.0.0/0.0.0.0:10020 connection refused

最新推荐文章于 2021-03-12 16:49:29 发布

转载最新推荐文章于 2021-03-12 16:49:29 发布 · 4.6k 阅读

本文介绍在Hadoop2.7.1环境中执行MapReduce程序时遇到连接异常问题的解决方案，包括配置文件mapred-site.xml的正确设置及启动MapReduce JobHistoryServer服务的方法。

出现问题描述:

在hadoop2.7.1 中执行MapReduce程序时出现下面的错误.

14/04/04 17:32:34 INFOmapred.ClientServiceDelegate: Application state is completed.FinalApplicationStatus=SUCCEEDED. Redirecting to job history server
14/04/04 17:32:35 INFO ipc.Client: Retryingconnect to server: 0.0.0.0/0.0.0.0:10020. Already tried 0 time(s); retry policyis RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
14/04/04 17:32:36 INFO ipc.Client: Retryingconnect to server: 0.0.0.0/0.0.0.0:10020. Already tried 1 time(s); retry policyis RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
...
java.io.IOException:java.net.ConnectException: Call From master/192.168.182.100 to 0.0.0.0:10020failed on connection exception: java.net.ConnectException: Connection refused;For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
         atorg.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:334)
         atorg.apache.hadoop.mapred.ClientServiceDelegate.getJobStatus(ClientServiceDelegate.java:419)
         atorg.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:532)
         atorg.apache.hadoop.mapreduce.Job$1.run(Job.java:314)
         atorg.apache.hadoop.mapreduce.Job$1.run(Job.java:311)
         atjava.security.AccessController.doPrivileged(Native Method)
         atjavax.security.auth.Subject.doAs(Subject.java:415)
         atorg.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
         ...

从错误信息中需要访问端口 10020 可以大概看出，DataNode 需要访问 MapReduce JobHistory Server，如果没有修改则用默认值：0.0.0.0:10020 。需要修改配置文件 mapred-site.xml ：

<property>
   <name>mapreduce.jobhistory.address</name>
   <!-- 配置实际的主机名和端口-->
   <value>master:10020</value>
</property>

并且启动MapReduce JobHistory Server服务:

在namenode上执行命令:

sbin/mr-jobhistory-daemon.sh start historyserver

这样在，namenode上会启动JobHistoryServer服务，可以在historyserver的日志中查看运行情况

参考地址:http://blog.csdn.net/tangzwgo/article/details/25893989

http://www.micmiu.com/bigdata/hive/hive-connect-exception-to-10020/