HADOOP

1、配置hadoop
[hadoop@server1 ~]$ cd hadoop/etc/
[hadoop@server1 etc]$ vim hadoop/slaves 
[hadoop@server1 etc]$ vim hadoop/core-site.xml 
[hadoop@server1 etc]$ vim hadoop/hdfs-site.xml 
[hadoop@server1 etc]$ cd ~/hadoop

##格式化hdfs集群
[hadoop@server1 hadoop]$ bin/hdfs namenode -format
[hadoop@server1 hadoop]$ scp -r /tmp/hadoop-hadoop 172.25.77.5:/tmp/

##查看server5主机
[root@server5 ~]# ls /tmp/
hadoop-hadoop

2、3个DN主机启动journalnod
[hadoop@server3 zookeeper-3.4.9]$ cd ~/hadoop
[hadoop@server3 hadoop]$ sbin/hadoop-daemon.sh start journalnode
starting journalnode, logging to /home/hadoop/hadoop-2.7.3/logs/hadoop-hadoop-journalnode-server3.out

查看3个DN主机zookeeper集群状态
[hadoop@server3 hadoop]$ jps
1881 DataNode
1698 QuorumPeerMain
1983 Jps
1790 JournalNode

3、NN主机格式化zookeeper
[hadoop@server1 hadoop]$ bin/hdfs zkfc -formatZK

##启动zookeeper
[hadoop@server1 hadoop]$ sbin/start-dfs.sh
Starting namenodes on [server1 server5]
server5: starting namenode, logging to /home/hadoop/hadoop-2.7.3/logs/hadoop-hadoop-namenode-server5.out
server1: starting namenode, logging to /home/hadoop/hadoop-2.7.3/logs/hadoop-hadoop-namenode-server1.out
172.25.77.4: starting datanode, logging to /home/hadoop/hadoop-2.7.3/logs/hadoop-hadoop-datanode-server4.out
172.25.77.2: starting datanode, logging to /home/hadoop/hadoop-2.7.3/logs/hadoop-hadoop-datanode-server2.out
172.25.77.3: starting datanode, logging to /home/hadoop/hadoop-2.7.3/logs/hadoop-hadoop-datanode-server3.out
Starting journal nodes [172.25.77.2 172.25.77.3 172.25.77.4]
172.25.77.3: journalnode running as process 1790. Stop it first.
172.25.77.4: journalnode running as process 1706. Stop it first.
172.25.77.2: journalnode running as process 1626. Stop it first.
Starting ZK Failover Controllers on NN hosts [server1 server5]
server1: starting zkfc, logging to /home/hadoop/hadoop-2.7.3/logs/hadoop-hadoop-zkfc-server1.out
server5: starting zkfc, logging to /home/hadoop/hadoop-2.7.3/logs/hadoop-hadoop-zkfc-server5.out

##查看zookeeper集群
[hadoop@server1 hadoop]$ jps
6694 Jps
6646 DFSZKFailoverController
6352 NameNode

[hadoop@server5 ~]$ jps
1396 DFSZKFailoverController
1298 NameNode
1484 Jps

4、测试高可用
##访问 http://172.25.77.1:50070/dfshealth.html#tab-overview 查看master
##此时,server5 为 master,server1 为 standby
[hadoop@server5 ~]$ jps
1396 DFSZKFailoverController
1298 NameNode
1484 Jps
[hadoop@server5 ~]$ kill -9 1298
[hadoop@server5 ~]$ jps
1396 DFSZKFailoverController
1515 Jps

##server1切换为 master
##server5再次启动,状态为 standby
[hadoop@server5 hadoop]$ sbin/hadoop-daemon.sh start namenode
starting namenode, logging to /home/hadoop/hadoop-2.7.3/logs/hadoop-hadoop-namenode-server5.out

5、DN主机查看:
[hadoop@server2 hadoop]$ cd ~/zookeeper-3.4.9
[hadoop@server2 zookeeper-3.4.9]$ bin/zkCli.sh
Connecting to localhost:2181
[zk: localhost:2181(CONNECTED) 3] ls /hadoop-ha/masters
[ActiveBreadCrumb, ActiveStandbyElectorLock]
[zk: localhost:2181(CONNECTED) 4] get /hadoop-ha/masters/Active

ActiveBreadCrumb           ActiveStandbyElectorLock
[zk: localhost:2181(CONNECTED) 4] get /hadoop-ha/masters/ActiveBreadCrumb
mastersh1server1 �F(�>
cZxid = 0x10000000a
ctime = 27 Aug  16:32:30 CST 2018
mZxid = 0x10000000e
mtime = 27 Aug 21 16:34:02 CST 2018
pZxid = 0x10000000a
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 28
numChildren = 0

阅读更多
想对作者说点什么?

博主推荐

换一批

没有更多推荐了,返回首页