1.报错时环境及配置
hadoop环境为一主三从,在namenode上启用yarn,配置yarn-site.xml,三个从节点yarn-site.xml配置文件为空。
yarn-site.xml的内容如下
<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
<property>
<name>yarn.resourcemanager.address</name>
<value>node01:8032</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>node01:8030</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>node01:8035</value>
</property>
<property>
<name>yarn.resourcemanager.admin.address</name>
<value>node01:8033</value>
</property>
</configuration>
2.错误集合
报错1:
输出信息卡死在“INFO mapreduce.Job: Running job: xxxxxxxxx”
错误重现:在执行hadoop自带实例程序,命令如下
yarn jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar pi 2 1
输出信息卡死在“INFO mapreduce.Job: Running job: xxxxxxxxx”
错误分析:打开从节点上的yarn相关日志文件,查看日志是否又错误,命令如下
vi hadoop/logs/yarn-root-nodemanager-node03.log
发现里面有如下问题
2019-10-18 23:21:36,604 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
解决方法:在每个从节点上配置与namenode相同的yarn-site.xml参数
报错2:
INFO mapreduce.Job: Job job_1571299226679_0001 failed with state FAILED due to: Application application_xxxxxx failed 2 times due to Error launching application_xxxxxx . Got exception: org.apache.hadoop.yarn.exceptions.YarnException: Unauthorized request to start container.
This token is expired. current time is 1571378108402 found 1571299940578
Note: System times on machines may be out of sync. Check system time and time zones.
这个错误的意思是 hadoop节点时间的时间不同步
解决方法:通过以下命令将每个节点的时间都设为一致
date -s '2019-10-18 22:49:00'