hadoop2 yarn 集群配置

一、配置文件配置

进入hadoop_home/etc/Hadoop目录

二、配置core_site.xml

<configuration>
        <property>
        <name>fs.defaultFS</name>
                <value>hdfs://xxxx:8020</value>
        </property>
        <property>
                <name>hadoop.tmp.dir</name>
                <value>/spark/data/hadoop/tmp</value>
                <description>A base for other temporary directories.</description>
        </property>
        <property>
                <name>io.file.buffer.size</name>
                <value>65535</value>
                <description>Size of read/write buffer used in SequenceFiles.</description>
        </property>
        <property>
                <name>io.native.lib.available</name>
                <value>true</value>
                <description>Should native hadoop libraries, if present, be used.</description>
        </property>
        <property>
                <name>fs.hdfs.impl</name>
                <value>org.apache.hadoop.hdfs.DistributedFileSystem</value>
                <description>The FileSystem for hdfs: uris.</description>
        </property>  
</configuration>

三、配置hadoop_env.sh

注意把里面的JAVA_HOME改为自已配的*注意要使用绝对路径

四、配置hdfs-site.xml

<configuration>
<property>
      <name>dfs.http.address</name>
            <value>xxxx:10075</value>
                <description>
                            The address and the base port where the dfs namenode web ui will listen on.
                            If the port is 0 then the server will start on a free port.
                </description>
        </property>
        <property>
            <name>dfs.namenode.secondary.http-address</name>
                <value>xxxx:10090</value>
        </property>
        <property>
            <name>dfs.namenode.name.dir</name>
                <value>/spark/data/hadoop/dfs/name</value>
        </property>
        <property>
            <name>dfs.datanode.data.dir</name>
                <value>/spark/data/hadoop/dfs/data</value>
        </property>
        <property>
            <name>dfs.replication</name>
                <value>3</value>
        </property>
        <property>
                <name>dfs.namenode.handler.count</name>
                <value>100</value>
        </property>
</configuration>

五、配置mapred-env.sh

注意把JAVA_HOME配置成自已的

六、配置slaves，即加入slave地址

七、配置yarn-env.sh

注意把JAVA_HOME配置成自已的

八、配置yarn-site.xml

<configuration>
<!-- Site specific YARN configuration properties -->
<property>
<name>yarn.resourcemanager.hostname</name>
<value>xxxx</value>
</property>
</configuration>

八、分发文件夹到其他主机上

九、格式化集群

bin/hadoop namenode -format

十、启动

十一、查看进程、查看日志