hadoop伪集群模式部署

系统：centOS6.5 x86_64

hadoop版本：hadoop-2.6.0-cdh5.7.0

JDK版本：1.7

参考网址：http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/SingleCluster.html

1、下载cdh版本的hadoop

http://archive.cloudera.com/cdh5/cdh/5/

2、解压、配置环境变量

tar -zxvf hadoop-2.6.0-cdh5.7.0-src.tar.gz

vi /etc/profile

HADOOP_HOME=/opt/software/hadoop

export PATH=${HADOOP_HOME}/bin:${HADOOP_HOME}/sbin:${PATH}

3、修改${HADOOP_HOME}/etc/hadoop 目录下的配置文件（下文中的hadoop001为本机的IP，通过修改/etc/hotss进行配置）

3.1 vi hadoop-env.sh

       export JAVA_HOME=/usr/java/jdk1.8.0_45

3.2 vi  core-site.xml

 <configuration>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://hadoop001:9000</value>
    </property>
        <property>
        <name>hadoop.tmp.dir</name>
        <value>/opt/software/hadoop-2.6.0-cdh5.7.0/data/tmp</value>
    </property>
</configuration>

3.3  vi  hdfs-site.xml

<configuration>
    <property>
        <name>dfs.namenode.name.dir</name>
        <value>/opt/software/hadoop-2.6.0-cdh5.7.0/data/namenode</value>
    </property>
    <property>
        <name>dfs.datanode.data.dir</name>
        <value>/opt/software/hadoop-2.6.0-cdh5.7.0/data/datanode</value>
    </property>


    <property>
        <name>dfs.replication</name>
        <value>3</value>
    </property>

    <property>
        <name>dfs.secondary.http.address</name>
        <value>hadoop001:50090</value>
    </property> 

    <property>
        <name>dfs.secondary.https.address</name>
        <value>hadoop001:50091</value>
    </property>

</configuration>

3.4vi mapred-site.xml

<configuration>
    <property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
    </property>
</configuration>

3.5 vi  yarn-site.xml

<configuration>
    <property>
        <name>yarn.resourcemanager.hostname</name>
        <value>hadoop001</value>
    </property>


    <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>
</configuration>

3.6 vi  salves

hadoop001

4设置ssh免密登陆

ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
chmod 0600 ~/.ssh/authorized_keys

5 启动集群(（确认已配置第二步中的环境变量）
初始化HDFS
hadoop  namenode  -format

启动HDFS
start-dfs.sh

启动YARN
start-yarn.sh

6最后通过jps命令检查即可。

[hadoop@hadoop001 etc]$ jps
18304 DataNode
19094 Jps
18186 NameNode
18699 NodeManager
18463 SecondaryNameNode
18607 ResourceManager

【来自@若泽大数据】