Hadoop 租约(一)

“租约” ，何为租约，一个租约管理着一个client的所有锁，对于每个client 都有一个相应的租约，但一个client 定期 check in 时租约的时间戳就会更新，当一个client 停止的时候并且允许租约到期，则对应的所有 锁都会释放。
大家来看看租约这个类：

class Lease implements Comparable<Lease> {
    // 租约的持有者，就是客户端名称 
    private final String holder;
   //最后一次检入时间
    private long lastUpdate;
    //持有的文件路径集
    private final Collection<String> paths = new TreeSet<String>();
  }

租约管理的超时限制：

• 软超时默认 60000ms=1min 当对一个文件进行创建或者append的时候会判断，配置 ：hdfs.regionserver.lease.period
• 硬超时默认时 60*60000ms=1h ,namenode 会监控硬链接是否超时，若超时会恢复租约

租约恢复算法

• Namenode检索租赁信息
• 对于租约中的每个文件f，使用f的最后一个块b
  • 获取包含b的datanode
  • 指定其中一个datanode 作为主要 的datanode 并命名其为p
  • p 从namenode 获取一个新生成的时间戳
  • p 从每个datanode 获取块信息
  • p 计算最小块的长度 length
  • p 使用新生成的时间戳更新和最小块长度 更新具有有效的生成时间戳的datanode
  • p 让namenode 更新信息
  • namenode 更新块信息
  • Namenode在删除所有文件后从租约中删除f并删除租约
    下面的这个方法会在客户端create或者append的时候执行该方法：

private void recoverLeaseInternal(INodeFile fileInode, 
      String src, String holder, String clientMachine, boolean force)
      throws IOException {
    assert hasWriteLock();
    if (fileInode != null && fileInode.isUnderConstruction()) {
      INodeFileUnderConstruction pendingFile = (INodeFileUnderConstruction) fileInode;
      /** 如果文件正在构建中，那么它必须在我们的租约中。找到合适的租赁记录。*/
      Lease lease = leaseManager.getLease(holder);
      //通过文件地址获取租约，若发现原创建该文件，会报异常
      if (!force && lease != null) {
        Lease leaseFile = leaseManager.getLeaseByPath(src);
        if ((leaseFile != null && leaseFile.equals(lease))) { 
          throw new AlreadyBeingCreatedException(
            "failed to create file " + src + " for " + holder +
            " on client " + clientMachine + 
            " because current leaseholder is trying to recreate file.");
        }
      }
      //
      // 获取原始的持有者      
      lease = leaseManager.getLease(pendingFile.getClientName());
      if (lease == null) {
        throw new AlreadyBeingCreatedException(
          "failed to create file " + src + " for " + holder +
          " on client " + clientMachine + 
          " because pendingCreates is non-null but no leases found.");
      }
      if (force) {
        //如果是强制的
        // 直接关闭：不需要 等待软超时过期
        // 只关闭当前的文件
        LOG.info("recoverLease: " + lease + ", src=" + src +
          " from client " + pendingFile.getClientName());
        internalReleaseLease(lease, src, holder);
      } else {
        assert lease.getHolder().equals(pendingFile.getClientName()) :
          "Current lease holder " + lease.getHolder() +
          " does not match file creator " + pendingFile.getClientName();
        //如果软超时已经超时 则尝试恢复租约
        if (lease.expiredSoftLimit()) {
          LOG.info("startFile: recover " + lease + ", src=" + src + " client "
              + pendingFile.getClientName());
          boolean isClosed = internalReleaseLease(lease, src, null);
          if(!isClosed)
            throw new RecoveryInProgressException(
                "Failed to close file " + src +
                ". Lease recovery is in progress. Try again later.");
        } else {
          final BlockInfo lastBlock = pendingFile.getLastBlock();
          if (lastBlock != null
              && lastBlock.getBlockUCState() == BlockUCState.UNDER_RECOVERY) {
            throw new RecoveryInProgressException("Recovery in progress, file ["
                + src + "], " + "lease owner [" + lease.getHolder() + "]");
          } else {
            throw new AlreadyBeingCreatedException("Failed to create file ["
                + src + "] for [" + holder + "] on client [" + clientMachine
                + "], because this file is already being created by ["
                + pendingFile.getClientName() + "] on ["
                + pendingFile.getClientMachine() + "]");
          }
        }
      }
    }
  }