hadoop-2.7.0集群部署

一、环境准备

准备三台服务器，分别编辑 vi /etc/hosts

192.168.37.242 node1
192.168.37.242 node2
192.168.37.242 node3

View Code

二、编码master(node1)配置

core-site.xml

<configuration>
    <property>
        <name>hadoop.tmp.dir</name>
        <value>file:/home/hadoop/tmp</value>
        <description>Abase for other temporary directories.</description>
    </property>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://node1:9000</value>
    </property>
        <property>
            <name>io.file.buffer.size</name>
            <value>131702</value>
        </property>
</configuration>

View Code

hdfs-site.xml

<configuration>
    <property>
        <name>dfs.replication</name>
        <value>3</value>
    </property>
    <property>
        <name>dfs.namenode.name.dir</name>
        <value>file:/home/hadoop/tmp/dfs/name</value>
    </property>
    <property>
        <name>dfs.datanode.data.dir</name>
        <value>file:/home/hadoop/tmp/dfs/data</value>
    </property>
        <property>
            <name>dfs.namenode.secondary.http-address</name>
            <value>node1:9001</value>
        </property>
        <property>
        <name>dfs.webhdfs.enabled</name>
        <value>true</value>
        </property>
</configuration>

View Code

mv mapred-site.xml.template mapred-site.xml

<configuration>
    <property>
            <name>mapreduce.framework.name</name>
            <value>yarn</value>
        </property>
        <property>
            <name>mapreduce.jobhistory.address</name>
            <value>node1:10020</value>
        </property>
        <property>
            <name>mapreduce.jobhistory.webapp.address</name>
            <value>node1:19888</value>
        </property>
</configuration>

View Code

yarn-site.xml

<configuration>

    <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>
    <property>
        <name>yarn.nodemanager.auxservices.mapreduce.shuffle.class</name>
        <value>org.apache.hadoop.mapred.ShuffleHandler</value>
    </property>
    <property>
        <name>yarn.resourcemanager.address</name>
        <value>node1:8032</value>
    </property>
    <property>
        <name>yarn.resourcemanager.scheduler.address</name>
        <value>node1:8030</value>
    </property>
    <property>
        <name>yarn.resourcemanager.resource-tracker.address</name>
        <value>node1:8031</value>
    </property>
    <property>
        <name>yarn.resourcemanager.admin.address</name>
        <value>node1:8033</value>
    </property>
    <property>
        <name>yarn.resourcemanager.webapp.address</name>
        <value>node1:8088</value>
    </property>
    <property>
        <name>yarn.nodemanager.resource.memory-mb</name>
        <value>768</value>
    </property>
</configuration>

View Code

修改etc/hadoop/slaves

#localhost
node2
node3

三、将node1安装文件复制到node2、node3

scp -r hadoop-2.7.0/ node2:/home/hadoop/
scp -r hadoop-2.7.0/ node3:/home/hadoop/

四、启动集群

1、主服务器(node1)上执行bin/hdfs namenode -format   进行初始化

2、sbin目录下执行 ./start-all.sh 

3、可以使用jps查看信息

4、web查看  http://192.168.37.242:8088/

5、停止的话，输入命令，sbin/stop-all.sh

转载于:https://my.oschina.net/u/3372233/blog/862819