HDFS源码分析(五)-----节点注册与心跳机制

前言

在Hadoop的HDFS启动的时候,不知道大家有没有注意到一个细节,一般都是先启动NameNode,然后再启动DataNode,细想一下,原因就很简单了,因为NameNode要维护元数据信息,而这些信息都是要等待后续启动的DataNode的情况汇报才能逐步构建的.然后之后通过保持心跳的形式进行block块映射关系的维护与更新.而今天的文章就以此方面,对这块流程做全面的分析.

相关涉及类

依旧需要介绍一下相关的涉及类,首先要有一个大概的了解.下面是主要的类:

1.DataNode--数据节点类,这个和之前的数据节点描述符类又又有点不同,里面也定义了许多与数据节点相关的方法.

2.NaemNode--名字节点类,注册信息的处理以及心跳包的处理都需要名字节点处理,名字节点的处理方法会调用FSNamesystem大系统中的方法.

3.DatanodeCommand以及BlockCommand--数据节点命令类以及他的子类,block相关命令类,此类用于名字节点心跳回复命令给数据节点时用的.

4.FSNamesystem和DatanodeDescriptor--附属类,这些类中的某些方法会在上述过程中被用到.

OK,涉及的类的总数也不多,下面讲述第一个流程,节点注册,数据节点是如何在启动的时候注册到名字节点的呢.

节点注册

节点的注册是在数据节点启动之后发生的,首先进入main主方法

public static void main(String args[]) {
    secureMain(args, null);
  }

然后进入secureMain

public static void secureMain(String [] args, SecureResources resources) {
    try {
      StringUtils.startupShutdownMessage(DataNode.class, args, LOG);
      DataNode datanode = createDataNode(args, null, resources);
      if (datanode != null)
        datanode.join();
    } catch (Throwable e) {
      LOG.error(StringUtils.stringifyException(e));
      System.exit(-1);
    } finally {
      // We need to add System.exit here because either shutdown was called or
      // some disk related conditions like volumes tolerated or volumes required
      // condition was not met. Also, In secure mode, control will go to Jsvc and
      // the process hangs without System.exit.
      LOG.info("Exiting Datanode");
      System.exit(0);
    }
  }

进入createDataNode方法,中间还会有1,2个方法,最终会调用到核心的runDatanodeDaemon()方法

/** Start a single datanode daemon and wait for it to finish.
   *  If this thread is specifically interrupted, it will stop waiting.
   * 数据节点启动的核心方法
   */
  public static void runDatanodeDaemon(DataNode dn) throws IOException {
    if (dn != null) {
      //register datanode
      //首先注册节点
      dn.register();
      //后续开启相应线程
      dn.dataNodeThread = new Thread(dn, dnThreadName);
      dn.dataNodeThread.setDaemon(true); // needed for JUnit testing
      dn.dataNodeThread.start();
    }
  }

于是就调用了register的方法

/**
   * Register datanode
   * <p>
   * The datanode needs to register with the namenode on startup in order
   * 1) to report which storage it is serving now and 
   * 2) to receive a registrationID 
   * issued by the namenode to recognize registered datanodes.
   * 
   * @see FSNamesystem#registerDatanode(DatanodeRegistration)
   * @throws IOException
   * 数据节点的注册方法,会调用到namenode上的注册方法
   */
  private void register() throws IOException {
    if (dnRegistration.getStorageID().equals("")) {
      setNewStorageID(dnRegistration);
    }
    while(shouldRun) {
      try {
        // reset name to machineName. Mainly for web interface.
        dnRegistration.name = machineName + ":" + dnRegistration.getPort();
        //调用namenode上的注册方法
        dnRegistration = namenode.register(dnRegistration);
        break;
....

然后这里,我们可以看到,注册方法实质上调用名字节点的注册方法.跟踪名字节点的同名法方法

  // DatanodeProtocol
  
  /** 
   * 名字节点的注册方法,调用的是FSNameSystem方法
   */
  public DatanodeRegistration register(DatanodeRegistration nodeReg
                                       ) throws IOException {
    //首先做版本验证 
    verifyVersion(nodeReg.getVersion());
    //调用namesystem的注册节点方法
    namesystem.registerDatanode(nodeReg);
      
    return nodeReg;
  }

名字节点调用的又是命名系统的方法，对注册节点的判断分为以下3种情况

1、现有的节点进行新的存储ID注册

2、现有节点的重复注册，由于集群已经保存有此信息，进行网络位置的更新即可

3、从未注册过的节点，直接进行分配新的存储ID进行注册。

具体方法判断如下，方法比较长

/**
   * Register Datanode.
   * <p>
   * The purpose of registration is to identify whether the new datanode
   * serves a new data storage, and will report new data block copies,
   * which the namenode was not aware of; or the datanode is a replacement
   * node for the data storage that was previously served by a different
   * or the same (in terms of host:port) datanode.
   * The data storages are distinguished by their storageIDs. When a new
   * data storage is reported the namenode issues a new unique storageID.
   * <p>
   * Finally, the namenode returns its namespaceID as the registrationID
   * for the datanodes. 
   * namespaceID is a persistent attribute of the name space.
   * The registrationID is checked every time the datanode is communicating
   * with the namenode. 
   * Datanodes with inappropriate registrationID are rejected.
   * If the namenode stops, and then restarts it can restore its 
   * namespaceID and will continue serving the datanodes that has previously
   * registered with the namenode without restarting the whole cluster.
   * 
   * @see org.apache.hadoop.hdfs.server.datanode.DataNode#register()
   * 名字节点实现数据节点的注册操作
   */
  public synchronized void registerDatanode(DatanodeRegistration nodeReg
                                            ) throws IOException {
    String dnAddress = Server.getRemoteAddress();
    if (dnAddress == null) {
      // Mostly called inside an RPC.
      // But if not, use address passed by the data-node.
      dnAddress = nodeReg.getHost();
    }      

    // check if the datanode is allowed to be connect to the namenode
    if (!verifyNodeRegistration(nodeReg, dnAddress)) {
      throw new DisallowedDatanodeException(nodeReg);
    }

    String hostName = nodeReg.getHost();
      
    // update the datanode's name with ip:port
    DatanodeID dnReg = new DatanodeID(dnAddress + ":" + nodeReg.getPort(),
                                      nodeReg.getStorageID(),
                                      nodeReg.getInfoPort(),
                                      nodeReg.getIpcPort());
    nodeReg.updateRegInfo(dnReg);
    nodeReg.exportedKeys = getBlockKeys();
      
    NameNod