hadoop2.7.2集群简易安装

hadoop2.7.2集群简易安装

1、下载hadoop2.7.2 hadoop-2.7.2 百度网盘

提取码：ht13  

2、安装zookeeper集群 zookeeper集群安装

也可以不安装zookeeper集群

3、解压hadoop2.7.2

tar -zxvf hadoop-2.7.2.tar.gz

##解压到其他目录
tar -zxvf hadoop-2.7.2.tar.gz -C [目录名称]

4、配置hadoop2.7.2

vi /home/software/hadoop-2.7.2/etc/hadoop

core-site.xml

<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://hadoop01:9000</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/opt/hadoop-2.7.2/data/tmp</value>
</property>
</configuration>

hdfs.site.xml

<configuration>
<property>
<name>dfs.replication</name>
<value>3</value>
</property>
<property>
<name>dfs.namenode.secondary.http-address</name>
<value>hadoop03:50090</value>
</property>
</configuration>

mapred-site.xml

<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>mapreduce.jobhistory.address</name>
<value>hadoop01:10020</value>
</property>
<property>
<name>mapreduce.jobhistory.webapp.address</name>
<value>hadoop01:19888</value>
</property>
</configuration>

yarn-site.xml

<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<!-- Site specific YARN configuration properties -->
<property>
<name>yarn.resourcemanager.hostname</name>
<value>hadoop02</value>
</property>
<property>
<name>yarn.log-aggregation-enable</name>
<value>true</value>
</property>
<property>
<name>yarn.log-aggregation.retain-seconds</name>
<value>604800</value>
</property>
</configuration>

hadopp-env.sh

export JAVA_=/opt/jdk1.8.0_181

mapred-env.sh

export JAVA_HOME=/opt/jdk1.8.0_181
export HADOOP_JOB_HISTORYSERVER_HEAPSIZE=1000
export HADOOP_MAPRED_ROOT_LOGGER=INFO,RFA

slaves

hadoop01
hadoop02
hadoop03

5、配置hadoop环境变量

export HADOOP_HOME=/home/software/hadoop2.7.2
export PATH=$HADOOP_HOME/bin:$PATH

6、格式化namenode

hdfs namenode -format

或

hadoop namenode -format

7、启动namenode

hadoop-deamon.sh start namenode

8、启动hadoop集群

1、单点启动
	启动hdfs

start-hdfs.sh start

	启动yarn

start-yarn.sh start

2、群起服务

start-all.sh start