Hadoop HA 高可用集群搭建配置（经过测试）

2019独角兽企业重金招聘Python工程师标准>>>

集群环境 准备

• 参考资料
  • High Avaliability With QJM
    • http://hadoop.apache.org/docs/r2.5.2/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithQJM.html
  • zookeeper 
    • http://zookeeper.apache.org/doc/r3.3.6/zookeeperStarted.html

	IP	Namenode	DataNode	Journalnode	Zookeeper	ZKFC	ResourceManager	NodeManager
master	192.168.1.200	1			1	1
slave1	192.168.1.201	1	1	1	1	1		1
slave2	192.168.1.202		1	1	1		1	1
slave3	192.168.1.203		1	1			1	1

 

网络配置 hosts 防火墙关闭

192.168.1.200 master
192.168.1.201 slave1
192.168.1.202 slave2
192.168.1.203 slave3

service iptables stop
chkconfig iptables off

时间同步(NN SNN DN 同步时间)

### s1a.time.du.cn 时间同步服务器
ntpdate s1a.time.du.cn

免密钥配置   两台NN之间一定要免密钥

ssh-keygen -t rsa（master主机上）
ssh-copy-id -i ~/.ssh/id_rsa.pub root@master（master主机上）
ssh-copy-id -i ~/.ssh/id_rsa.pub root@slave1（master主机上）
ssh-copy-id -i ~/.ssh/id_rsa.pub root@slave2（master主机上）
ssh-copy-id -i ~/.ssh/id_rsa.pub root@slave3（master主机上）

ssh-keygen -t rsa（slave1主机上）
ssh-copy-id -i ~/.ssh/id_rsa.pub root@master（slave1主机上）
ssh-copy-id -i ~/.ssh/id_rsa.pub root@slave1（slave1主机上）
ssh-copy-id -i ~/.ssh/id_rsa.pub root@slave2（slave1主机上）
ssh-copy-id -i ~/.ssh/id_rsa.pub root@slave3（slave1主机上）

###测试访问
ssh master

jdk 配置环境变量

#cd /tmp/
#tar xf hadoop-2.5.1_x64.tar.gz
#mv hadoop-2.5.1 /opt/hadoop
#tar xf zookeeper-3.4.6.tar.gz.tar.gz
#mv zookeeper-3.4.6 /opt/zookeeper
#vim /etc/profile
export HADOOP_HOME=/opt/hadoop
export PATH=$PATH:$HADOOP_HOME/bin:$PATH:$HADOOP_HOME/sbin

export ZOOKEEPER_HOME=/opt/zookeeper
export PATH=$PATH:$ZOOKEEPER_HOME/bin

修改各项配置文件

###etc/hadoop/core-site.xml:
<configuration>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://mycluster</value>
    </property>
    <property>
	<name>hadoop.tmp.dir</name>
        <value>/opt/hdfs</value>
    </property>
<property>
   <name>ha.zookeeper.quorum</name>
   <value>master:2181,slave1:2181,slave2:2181</value>
 </property>
</configuration>
	
###etc/hadoop/hdfs-site.xml:
<configuration>
<property>
  <name>dfs.nameservices</name>
  <value>mycluster</value>
</property>

<property>
  <name>dfs.ha.namenodes.mycluster</name>
  <value>nn1,nn2</value>
</property>

<property>
  <name>dfs.namenode.rpc-address.mycluster.nn1</name>
  <value>master:8020</value>
</property>
<property>
  <name>dfs.namenode.rpc-address.mycluster.nn2</name>
  <value>slave1:8020</value>
</property> 

<property>
  <name>dfs.namenode.http-address.mycluster.nn1</name>
  <value>master:50070</value>
</property>
<property>
  <name>dfs.namenode.http-address.mycluster.nn2</name>
  <value>slave1:50070</value>
</property>

<property>
  <name>dfs.namenode.shared.edits.dir</name>
  <value>qjournal://slave1:8485;slave2:8485;slave3:8485/mycluster</value>
</property>

<property>
  <name>dfs.client.failover.proxy.provider.mycluster</name>
  <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
</property>

<property>
  <name>dfs.ha.fencing.methods</name>
  <value>sshfence</value>
</property>

<property>
  <name>dfs.ha.fencing.ssh.private-key-files</name>
  <value>/root/.ssh/id_rsa</value>
</property>

<property>
  <name>dfs.journalnode.edits.dir</name>
  <value>/opt/journal/data</value>
</property>

<property>
  <name>dfs.ha.automatic-failover.enabled</name>
  <value>true</value>
</property>
</configuration>

###mapred.site.xml
<configuration>
<property>
        <name>mapreduce.framework.name</name>
       <value>yarn</value>
 </property>
</configuration>

###yarn-site.xml
<configuration>

<!-- Site specific YARN configuration properties -->
 <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>

<property>
   <name>yarn.resourcemanager.ha.enabled</name>
   <value>true</value>
 </property>
 <property>
   <name>yarn.resourcemanager.cluster-id</name>
   <value>cluster1</value>
 </property>
 <property>
   <name>yarn.resourcemanager.ha.rm-ids</name>
   <value>rm1,rm2</value>
 </property>
 <property>
   <name>yarn.resourcemanager.hostname.rm1</name>
   <value>slave2</value>
 </property>
 <property>
   <name>yarn.resourcemanager.hostname.rm2</name>
   <value>slave3</value>
 </property>
 <property>
   <name>yarn.resourcemanager.zk-address</name>
   <value>master:2181,slave1:2181,slave2:2181</value>
 </property>

</configuration>
-----------------------------------------------------------------------------------
###zookeeper配置(在安装zookeeper的机子上 master slave1 slave2)
vim /opt/zookeeper/conf/zoo.cfg
tickTime=2000
dataDir=/var/zookeeper
clientPort=2181
initLimit=5
syncLimit=2
server.1=master:2888:3888
server.2=slave1:2888:3888
server.3=slave2:2888:3888

###在(在安装zookeeper的机子上 master slave1 slave2)创建目录/var/zookeeper
###master
vim /var/zookeeper/myid
1
###slave1 
vim /var/zookeeper/myid
2
###slave2
vim /var/zookeeper/myid
3

#vim etc/hadoop/masters--手动创建(完全分布式创建，HA下需要删除：没有SecondaryNamenode这个概念)
slave1
	
#vim etc/hadoop/slaves-手动创建
slave1
slave2
slave3

同步配置文件然后格式化

###同步master机器上hadoop所有配置文件（*）
scp -r /opt/hadoop-2.5.1/ root@slave1:/opt/
scp -r /opt/hadoop-2.5.1/ root@slave2:/opt/
scp -r /opt/hadoop-2.5.1/ root@slave3:/opt/

###启动JN集群 分别启动每一台JN hadoop-daemon.sh start journalnode
###在一台NN上执行格式化 hdfs namenode -format  启动当前NN hadoop-daemon.sh start namenode
###在另外一台NN上执行同步 hdfs namenode -bootstrapStandby

###格式化ZK
###在一台NN上 执行格式化  hdfs zkfc -formatZK

###启动hdfs集群
#start-dfs.sh

通过浏览器访问：master:50070

 

转载于:https://my.oschina.net/rosetta/blog/747135