Hadoop搭建高可用集群,出现Container exited with a non-zero exit code 1. Error file: prelaunch.err.

前情提要:
搭建好Hadoop HA集群后,试着跑一个wordcount,
结果居然出了这个问题:

Container exited with a non-zero exit code 1. Error file: prelaunch.err.

捏🐴,怎么回事?之前伪分布式或者非高可用的完全分布式好像没有这个问题啊?

于是百度,噢,原来是mapred-site.xml和yarn-site.xml里要加上classpath

输入hadoop classpath,出来一大串

在这里插入图片描述

然后复制这一大串,到mapred-site.xml和yarn-site.xml上

mapred-site.xml:

<property> 
    <name>mapreduce.application.classpath</name>
    <value>/home/hadoop/app/hadoop/etc/hadoop:/home/hadoop/app/hadoop/share/hadoop/common/lib/*:/home/hadoop/app/hadoop/share/hadoop/common/*:/home/hadoop/app/hadoop/share/hadoop/hdfs:/home/hadoop/app/hadoop/share/hadoop/hdfs/lib/*:/home/hadoop/app/hadoop/share/hadoop/hdfs/*:/home/hadoop/app/hadoop/share/hadoop/mapreduce/*:/home/hadoop/app/hadoop/share/hadoop/yarn:/home/hadoop/app/hadoop/share/hadoop/yarn/lib/*:/home/hadoop/app/hadoop/share/hadoop/yarn/*
</value>
</property>

yarn-site.xml

 <property>
    <name>yarn.application.classpath</name>
    <value>/home/hadoop/app/hadoop/etc/hadoop:/home/hadoop/app/hadoop/share/hadoop/common/lib/*:/home/hadoop/app/hadoop/share/hadoop/common/*:/home/hadoop/app/hadoop/share/hadoop/hdfs:/home/hadoop/app/hadoop/share/hadoop/hdfs/lib/*:/home/hadoop/app/hadoop/share/hadoop/hdfs/*:/home/hadoop/app/hadoop/share/hadoop/mapreduce/*:/home/hadoop/app/hadoop/share/hadoop/yarn:/home/hadoop/app/hadoop/share/hadoop/yarn/lib/*:/home/hadoop/app/hadoop/share/hadoop/yarn/*
</value>
</property>

(value里改成自己出来的那个classpath)



那么改完之后就完事了吗?

当然是并没有。

那么继续找,噢,原来是hadoop高版本需要配置yarn对外暴露的端口:

<!--配置rm2--> 
<property> 
  <name>yarn.resourcemanager.address.rm2</name> 
  <value>worker1:8132</value> 
</property> 
<property> 
  <name>yarn.resourcemanager.scheduler.address.rm2</name> 
  <value>worker1:8130</value> 
</property> 
<property> 
  <name>yarn.resourcemanager.webapp.address.rm2</name> 
  <value>worker1:8188</value> 
</property> 
<property> 
  <name>yarn.resourcemanager.resource-tracker.address.rm2</name> 
  <value>worker1:8131</value> 
</property> 
<property> 
  <name>yarn.resourcemanager.admin.address.rm2</name> 
  <value>worker1:8133</value> 
</property> 
<property> 
  <name>yarn.resourcemanager.ha.admin.address.rm2</name> 
  <value>worker1:23142</value> 
</property> 

<!--配置rm3--> 
<property> 
  <name>yarn.resourcemanager.address.rm3</name> 
  <value>worker2:8132</value> 
</property> 
<property> 
  <name>yarn.resourcemanager.scheduler.address.rm3</name> 
  <value>worker2:8130</value> 
</property> 
<property> 
  <name>yarn.resourcemanager.webapp.address.rm3</name> 
  <value>worker2:8188</value> 
</property> 
<property> 
  <name>yarn.resourcemanager.resource-tracker.address.rm3</name> 
  <value>worker2:8131</value> 
</property> 
<property> 
  <name>yarn.resourcemanager.admin.address.rm3</name> 
  <value>worker2:8133</value> 
</property> 
<property> 
  <name>yarn.resourcemanager.ha.admin.address.rm3</name> 
  <value>worker2:23142</value> 
</property> 
 

那么我个人的完整yarn-site.xml如下:

<?xml version="1.0"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->
<configuration>
    <!-- 开启RM高可用 -->
    <property>
        <name>yarn.resourcemanager.ha.enabled</name>
        <value>true</value>
    </property>

    <!-- 指定RM的cluster id -->
    <property>
        <name>yarn.resourcemanager.cluster-id</name>
        <value>yrc</value>
    </property>

    <!-- 指定RM的名字 -->
    <property>
        <name>yarn.resourcemanager.ha.rm-ids</name>
        <value>rm2,rm3</value>
    </property>

    <!-- 分别指定RM的地址 -->
 
    <property>
        <name>yarn.resourcemanager.hostname.rm2</name>
        <value>worker1</value>
    </property>

    <property>
        <name>yarn.resourcemanager.hostname.rm3</name>
        <value>worker2</value>
    </property>
 

<property>
 <name>yarn.log.server.url</name>
 <value>http://master:19888/jobhistory/logs</value>
</property>


<!--配置rm2--> 
<property> 
  <name>yarn.resourcemanager.address.rm2</name> 
  <value>worker1:8132</value> 
</property> 
<property> 
  <name>yarn.resourcemanager.scheduler.address.rm2</name> 
  <value>worker1:8130</value> 
</property> 
<property> 
  <name>yarn.resourcemanager.webapp.address.rm2</name> 
  <value>worker1:8188</value> 
</property> 
<property> 
  <name>yarn.resourcemanager.resource-tracker.address.rm2</name> 
  <value>worker1:8131</value> 
</property> 
<property> 
  <name>yarn.resourcemanager.admin.address.rm2</name> 
  <value>worker1:8133</value> 
</property> 
<property> 
  <name>yarn.resourcemanager.ha.admin.address.rm2</name> 
  <value>worker1:23142</value> 
</property> 

<!--配置rm3--> 
<property> 
  <name>yarn.resourcemanager.address.rm3</name> 
  <value>worker2:8132</value> 
</property> 
<property> 
  <name>yarn.resourcemanager.scheduler.address.rm3</name> 
  <value>worker2:8130</value> 
</property> 
<property> 
  <name>yarn.resourcemanager.webapp.address.rm3</name> 
  <value>worker2:8188</value> 
</property> 
<property> 
  <name>yarn.resourcemanager.resource-tracker.address.rm3</name> 
  <value>worker2:8131</value> 
</property> 
<property> 
  <name>yarn.resourcemanager.admin.address.rm3</name> 
  <value>worker2:8133</value> 
</property> 
<property> 
  <name>yarn.resourcemanager.ha.admin.address.rm3</name> 
  <value>worker2:23142</value> 
</property> 

    <!-- 指定zk集群地址 -->
    <property>
        <name>yarn.resourcemanager.zk-address</name>
        <value>master:2181,worker1:2181,worker2:2181</value>
    </property>

    <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>

    <property>
        <name>yarn.log-aggregation-enable</name>
        <value>true</value>
    </property>

    <property>
        <name>yarn.log-aggregation.retain-seconds</name>
        <value>86400</value>
    </property>

    <!-- 启用自动恢复 -->
    <property>
        <name>yarn.resourcemanager.recovery.enabled</name>
        <value>true</value>
    </property>

    <!-- 制定resourcemanager的状态信息存储在zookeeper集群上 -->
    <property>
        <name>yarn.resourcemanager.store.class</name>
        <value>org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore</value>
    </property>

  <property>
    <name>yarn.application.classpath</name>
    <value>/home/hadoop/app/hadoop/etc/hadoop:/home/hadoop/app/hadoop/share/hadoop/common/lib/*:/home/hadoop/app/hadoop/share/hadoop/common/*:/home/hadoop/app/hadoop/share/hadoop/hdfs:/home/hadoop/app/hadoop/share/hadoop/hdfs/lib/*:/home/hadoop/app/hadoop/share/hadoop/hdfs/*:/home/hadoop/app/hadoop/share/hadoop/mapreduce/*:/home/hadoop/app/hadoop/share/hadoop/yarn:/home/hadoop/app/hadoop/share/hadoop/yarn/lib/*:/home/hadoop/app/hadoop/share/hadoop/yarn/*
</value>
</property>



</configuration>

修改完毕,问题总算解决了。
网上那些让改classpath的,还是改环境变量的,还是都改的,真的让我改到怀疑人生~~

  • 4
    点赞
  • 22
    收藏
    觉得还不错? 一键收藏
  • 2
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 2
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值