Hadoop 2.x 伪分布式环境部署

1、 安装准备

a)        解压 tar –zxvf

b)       目录：/opt/modules/hadoop-2.2.0

c)        关闭防火墙

                       i.             service iptables status

                     ii.                                  service iptables stop

                   iii.                                  chkconfig iptables off

d)       关闭SELINUX

                       i.             vi /etc/sysconfig/selinux         

                     ii.                                  设置 SELINUX=disabled

e)        设置静态IP

                       i.             more/etc/sysconfig/network-scripts/ifcfg-eth0

f)        修改hostname

                       i.             more/etc/sysconfig/network-scripts/ifcfg-eth0

g)       IP与HostName绑定

                       i.             vim /etc/hosts

192.168.6.180hadoop-yarn.dragon.org hadoop-yarn

h)       安装JDK及配置环境变量

                       i.             /opt/modules/jdk1.6.0_45

 

2、 解压hadoop（编译后的）

tar -zxvf hadoop-2.2.0.tar.gz -C /opt/modules/

/opt/modules/hadoop-2.2.0

3、 配置

a)        路径：/opt/modules/hadoop-2.2.0/etc/hadoop

b)       hadoop-env.sh

                       i.             exportJAVA_HOME=/opt/modules/jdk1.6.0_45

c)        yarn.env.sh

                       i.             exportJAVA_HOME=/opt/modules/jdk1.6.0_45

d)       mapred-env.sh

                       i.             exportJAVA_HOME=/opt/modules/jdk1.6.0_45

e)        core-site.xml

<configuration>        <property>                                        <name>fs.default.name</name>                             <value>hdfs://hadoop-yarn.dragon.org:8020</value>        </property>          <property>                             <name>hadoop.tmp.dir</name>                             <value>/opt/modules/hadoop-2.2.0/data/tmp</value>        </property> </configuration>

                       i.              

f)        hdfs-site.xml

<configuration>        <property>                                        <name>dfs.replication</name>                             <value>1</value>        </property> </configuration>

 

g)       yarn-site.xml

<configuration>   <!-- Site specific YARN configuration properties -->        <property>                             <name>yarn.nodemanager.aux-services</name>                             <value>mapreduce_shuffle</value>        </property> </configuration>

 

h)       mapred-site.xml

cpmapred-site.xml.template mapred-site.xm

<configuration>        <property>                                                            <name>mapreduce.framework.name</name>                             <value>yarn</value>        </property> </configuration>

 

 

4、 启动

a)        启动HDFS

NameNode、DataNode、SecondaryNameNode

NameNode格式化：

cd  /opt/modules/hadoop-2.2.0/bin hdfs namenode -format

启动NameNode

cd /opt/modules/hadoop-2.2.0/sbin ./hadoop-daemon.sh start namenode

启动DataNode

cd /opt/modules/hadoop-2.2.0/sbin ./hadoop-daemon.sh start datanode

访问：http://hadoop-yarn.dragon.org:50070/dfshealth.jsp

 

启动SecondaryNameNode

cd /opt/modules/hadoop-2.2.0/sbin ./hadoop-daemon.sh start secondarynamenode

 

访问：http://hadoop-yarn.dragon.org:50090/status.jsp

 

停止HDFS，按照上述启动顺序依次停止：

./hadoop-daemon.sh stop namenode ./hadoop-daemon.sh stop namenode ./hadoop-daemon.sh stop secondarynamenode

 

b)        启动YARN

                       i.             启动ResourceManager

./yarn-daemon.sh start resourcemanager

 

                     ii.             启动NodeManager

./yarn-daemon.sh start nodemanager

访问：http://hadoop-yarn.dragon.org:8088/cluster

 

                   iii.             停止YARN

./yarn-daemon.sh stop resourcemanager ./yarn-daemon.sh stop nodemanager

 

其他的启动方式：

start-dfs.sh
start-yarn.sh 

对应的停止方式

stop-dfs.sh 
stop-yarn.sh

还有一种启动方式

start-all.sh

对应的停止方式

stop-all.sh