参考文档:
https://yq.aliyun.com/articles/335669?spm=a2c4e.11155435.0.0.61fd5f09CD5EDz
https://blog.51cto.com/sstudent/1392447
前提实验:本实验是基于Federation, Zookeeper和HA这三个实验的综合Hadoop集群部署实验,在前提实验完成的基础上继续进行HDFS Federation+HA(NameNode Federation+HA)的部署实验
实验目标:部署一个小型的HDFS Federation+HA(NameNode Federation+HA)分布式高可用集群 ,用于教学演示
环境要求: 虚拟机VirtualBox 操作系统 Centos7
hadoop版本 hadoop-2.6.0-cdh5.7.0 ZooKeeper版本 zookeeper-3.4.5-cdh5.7.0.tar.gz
网络设置: Vitual Box的虚拟机网卡设置为桥接模式,IP地址和机房的网络处于同一网段,确保7台虚拟机互相能Ping通
如果单台实体机的内存容量能承载得了7台Linux虚拟机,则无需用两台实体机搭建桥接模式,使用仅主机模式即可
集群规划一(单台实体机内存不足,桥接模式):
实体机 | 虚拟机IP | 虚拟机主机名 | 集群角色(运行的进程) |
---|---|---|---|
PC1 | 10.100.235.200 | master1 | NameNode(Active) ,DFSZKFailoverController |
PC1 | 10.100.235.201 | standby1 | NameNode(Standby), DFSZKFailoverController |
PC1 | 10.100.235.202 | master2 | NameNode(Active) ,DFSZKFailoverController |
PC1 | 10.100.235.203 | standby2 | NameNode(Standby), DFSZKFailoverController |
PC2 | 10.100.235.204 | slave1 | DataNode, JournalNode, QuorumPeerMain |
PC2 | 10.100.235.205 | slave2 | DataNode , JournalNode, QuorumPeerMain |
PC2 | 10.100.235.207 | slave3 | DataNode , JournalNode, QuorumPeerMain |
集群规划二(单台实体机内存充足,仅主机模式):
实体机 | 虚拟机IP | 虚拟机主机名 | 集群角色(运行的进程) |
---|---|---|---|
PC1 | 192.168.56.9 | master1 | NameNode(Active) ,DFSZKFailoverController |
PC1 | 192.168.56.10 | standby1 | NameNode(Standby) ,DFSZKFailoverController |
PC1 | 192.168.56.11 | master2 | NameNode(Active) ,DFSZKFailoverController |
PC1 | 192.168.56.12 | standby2 | NameNode(Standby), DFSZKFailoverController |
PC1 | 192.168.56.13 | slave1 | DataNode, JournalNode, QuorumPeerMain |
PC1 | 192.168.56.14 | slave2 | DataNode, JournalNode, QuorumPeerMain |
PC1 | 192.168.56.15 | slave3 | DataNode, JournalNode, QuorumPeerMain |
实验步骤:
1)在PC1的Virtual Box复制四台虚拟