ccah-500 第47题 What is the purpose of ZooKeeper in such a configuration

47.You decide to create a cluster which runs HDFS in High Availability mode with automatic failover using Quorum Storage. What is the purpose of ZooKeeper in such a configuration?

A. It only keeps track of which NameNode is Active at any given time

B. It monitors an NFS mount point and reports if the mount point disappears

C. It both keeps track of which NameNode is Active at any given time, and manages the Edits file. Which is a log of changes to the HDFS filesystem

D. It only manages the Edits file, which is log of changes to the HDFS filesystem

E. Clients connect to ZooKeeper to determine which NameNode is Active

Answer: A

Reference:

stackoverflow:

Think of it like this, zookeeper is a group of people, each assigned to watch over a factory and coordinate them, journal node is a place where all factory managers can check others status and coordinate. QJM is a combination of both to be used in HA for better coordination in case of fail over.

zookeeper coordinates hbase regionservers and other hadoop modules which require zookeeper.

journal node coordinates hadoop datanodes with the namenode.

QJM coordinates regionservers using the technique used by journal node

on core hadoop setup only journal node is necessary in case of distributed setup

 

journalnode的作用是在HA的两个namenode之间保持editlog的共享同步。
zookeeper用于两个namenode之间互相的错误感知（active的掉了，standby的可以看见）。

 

oreily:

There are two choices for the highly available shared storage: an NFS filer, or a quorum journal manager (QJM). The QJM is a dedicated HDFS implementation, designed for the sole purpose of providing a highly available edit log, and is the recommended choice for most HDFS installations. The QJM runs as a group of journal nodes, and each edit must be written to a majority of the journal nodes. Typically, there are three journal nodes, so the system can tolerate the loss of one of them. This arrangement is similar to the way ZooKeeper works, although it is important to realize that the QJM implementation does not use ZooKeeper. (Note, however, that HDFS HA does use ZooKeeper for electing the active namenode, as explained in the next section.)

 

 

http://www.cloudera.com/content/cloudera-content/cloudera-

docs/CDH4/latest/PDF/CDH4-High-Availability-Guide.pdf(page 15)

 edit file is maintain by journal node.Once active name node down zookeeper is responsilbe for failover however journal maintain edit log file,stand by name node refer this jounal node file once it become active.

 

http://www.cloudera.com/documentation/cdh/5-0-x/CDH5-High-Availability-Guide/cdh5hag_hdfs_ha_software_config.html

Automatic failover adds two new components to an HDFS deployment: a ZooKeeper quorum, and the ZKFailoverController process (abbreviated as ZKFC).

Apache ZooKeeper is a highly available service for maintaining small amounts of coordination data, notifying clients of changes in that data, and monitoring clients for failures. The implementation of automatic HDFS failover relies on ZooKeeper for the following things:

Failure detection - each of the NameNode machines in the cluster maintains a persistent session in ZooKeeper. If the machine crashes, the ZooKeeper session will expire, notifying the other NameNode that a failover should be triggered.

Active NameNode election - ZooKeeper provides a simple mechanism to exclusively elect a node as active. If the current active NameNode crashes, another node can take a special exclusive lock in ZooKeeper indicating that it should become the next active NameNode.

The ZKFailoverController (ZKFC) is a new component - a ZooKeeper client which also monitors and manages the state of the NameNode. Each of the machines which runs a NameNode also runs a ZKFC, and that ZKFC is responsible for:

Health monitoring - the ZKFC pings its local NameNode on a periodic basis with a health-check command. So long as the NameNode responds promptly with a healthy status, the ZKFC considers the node healthy. If the node has crashed, frozen, or otherwise entered an unhealthy state, the health monitor will mark it as unhealthy.

ZooKeeper session management - when the local NameNode is healthy, the ZKFC holds a session open in ZooKeeper. If the local NameNode is active, it also holds a special lock znode. This lock uses ZooKeeper's support for "ephemeral" nodes; if the session expires, the lock node will be automatically deleted.

ZooKeeper-based election - if the local NameNode is healthy, and the ZKFC sees that no other node currently holds the lock znode, it will itself try to acquire the lock. If it succeeds, then it has "won the election", and is responsible for running a failover to make its local NameNode active. The failover process is similar to the manual failover described above: first, the previous active is fenced if necessary, and then the local NameNode transitions to active state.