hadoop2.7.1集群搭建（虚拟机）--配置文件

最新推荐文章于 2023-05-22 16:15:18 发布

iteye_12229

最新推荐文章于 2023-05-22 16:15:18 发布

阅读量210

点赞数

分类专栏： hadoop 文章标签：运维大数据 java

本文链接：https://blog.csdn.net/iteye_12229/article/details/82651295

版权

hadoop 专栏收录该内容

3 篇文章 0 订阅

订阅专栏

一、core-site.xml
hadoop用户为fulndon:fulndon namenode结点为master


   <property>
        <name>fs.defaultFS</name>
        <value>hdfs://master:9000</value>
    </property>
    <property>
        <name>io.file.buffer.size</name>
        <value>131702</value>
    </property>
    <property>
        <name>hadoop.tmp.dir</name>
        <value>file:/home/fulndon/hadoop-2.7.1/tmp</value>
        <description>Abase for other temporary directories.</description>
    </property>
    <property>
        <name>hadoop.proxyuser.fulndon.hosts</name>
        <value>*</value>
    </property>
    <property>
        <name>hadoop.proxyuser.fulndon.groups</name>
        <value>*</value>
    </property>

二、hdfs-site.xml


  <property>
        <name>dfs.namenode.name.dir</name>
        <value>file:/home/fulndon/hadoop-2.7.1/hdfs/name</value>
    </property>
    <property>
        <name>dfs.datanode.data.dir</name>
        <value>file:/home/fulndon/hadoop-2.7.1/hdfs/data</value>
    </property>
    <property>
        <name>dfs.replication</name>
        <value>3</value>
    </property>
    <property>
        <name>dfs.namenode.secondary.http-address</name>
        <value>master:9001</value>
    </property>
    <property>
        <name>dfs.webhdfs.enabled</name>
        <value>true</value>
    </property>

三、mapred-site.xml.template


 <property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
    </property>

四、yarn-site.xml


 <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>
    <property>
        <name>yarn.nodemanager.auxservices.mapreduce.shuffle.class</name>
        <value>org.apache.hadoop.mapred.ShuffleHandler</value>
    </property>
    <property>
        <name>yarn.resourcemanager.address</name>
        <value>master:8032</value>
    </property>
    <property>
        <name>yarn.resourcemanager.scheduler.address</name>
        <value>master:8030</value>
    </property>
    <property>
        <name>yarn.resourcemanager.resource-tracker.address</name>
        <value>master:8031</value>
    </property>
    <property>
        <name>yarn.resourcemanager.admin.address</name>
        <value>master:8033</value>
    </property>
    <property>
        <name>yarn.resourcemanager.webapp.address</name>
        <value>master:8088</value>                                              
    </property>

五、向hadoop-env.sh和yarn-env.sh中追加jdk路径
// echo export JAVA_HOME=$JAVA_HOME >> yarn-env.sh
echo export JAVA_HOME=$JAVA_HOME | tee -a yarn-env.sh -a hadoop-env.sh