HBase源码阅读（三）如何定位region

上一篇文章中介绍了HTable中读写数据的逻辑，本篇介绍一下client如何在hbase集群中定位到需要的region。先介绍一下大致原理，1.2版本中的所有region布局信息被存放在meta表中，meta表永不分裂，仅有一个region，meta表的的位置被存放在zk中，所以client想要找到一个region的位置，需要先访问zk，找到meta表的位置，然后在meta表中查询到自己需要访问的region的信息。

meta表中信息如下图所示（图片引用自《HBase原理与实践》）：

可以看到，meta表中也存放的是key-value，键是TableName+StartRow+TimeStamp+EncodedName，值就是region的各种信息，包括region所在的server信息。下面开始看代码，在HTable的读写流程中都需要定位region的位置，都会调用ConnectionManager的locateRegion函数

public RegionLocations locateRegion(final TableName tableName,
      final byte [] row, boolean useCache, boolean retry, int replicaId)
    throws IOException {
      if (this.closed) throw new IOException(toString() + " closed");
      if (tableName== null || tableName.getName().length == 0) {
        throw new IllegalArgumentException(
            "table name cannot be null or zero length");
      }
      if (tableName.equals(TableName.META_TABLE_NAME)) {
        // 获取meta表信息
        return locateMeta(tableName, useCache, replicaId);
      } else {
        // Region not in the cache - have to go to the meta RS
        // 定位其他表的位置信息
        return locateRegionInMeta(tableName, row, useCache, retry, replicaId);
      }
    }

该函数实现非常简单，继续往下，我们看看如何获取meta表的位置：

    private RegionLocations locateMeta(final TableName tableName,
        boolean useCache, int replicaId) throws IOException {
      // HBASE-10785: We cache the location of the META itself, so that we are not overloading
      // zookeeper with one request for every region lookup. We cache the META with empty row
      // key in MetaCache.
      byte[] metaCacheKey = HConstants.EMPTY_START_ROW; // use byte[0] as the row for meta
      RegionLocations locations = null;
      if (useCache) {
        // 如果缓存中存在，就在缓存拿
        // 缓存就是个ConcurrentMap
        // TableName => {StartKey => regionInfo}
        locations = getCachedLocation(tableName, metaCacheKey);
        if (locations != null && locations.getRegionLocation(replicaId) != null) {
          return locations;
        }
      }

      // only one thread should do the lookup.
      // single flight逻辑，一个去访问zk就行，不要并发访问zk
      synchronized (metaRegionLock) {
        // Check the cache again for a hit in case some other thread made the
        // same query while we were waiting on the lock.
        if (useCache) {
          locations = getCachedLocation(tableName, metaCacheKey);
          if (locations != null && locations.getRegionLocation(replicaId) != null) {
            return locations;
          }
        }

        // Look up from zookeeper
        // 在zk中查找数据
        locations = this.registry.getMetaRegionLocation();
        if (locations != null) {
          // 将zk中查到的数据放到缓存中
          cacheLocation(tableName, locations);
        }
      }
      return locations;
    }

这个函数的逻辑也非常简单，如果缓存中有，就在缓存中拿meta表的路有信息，缓存就是个大map，映射关系为TableName => {StartKey => regionInfo}，如果没有，就去zk中拿meta表的信息，由getMetaRegionLocation实现，访问zk的细节就没继续看了，以后用到了再看吧。回过头，继续看如何定位其他region所在的位置，该功能由ConnectionManager的locateRegionInMeta函数实现。

   private RegionLocations locateRegionInMeta(TableName tableName, byte[] row,
                   boolean useCache, boolean retry, int replicaId) throws IOException {

      // If we are supposed to be using the cache, look in the cache to see if
      // we already have the region.
      if (useCache) {
        RegionLocations locations = getCachedLocation(tableName, row);
        if (locations != null && locations.getRegionLocation(replicaId) != null) {
          return locations;
        }
      }
      // 创建metaKey，根据metaKey来获取对应的region信息
      // meta key是 table_row_maxTime
      byte[] metaKey = HRegionInfo.createRegionName(tableName, row, HConstants.NINES, false);

      Scan s = new Scan();
      s.setReversed(true);
      // 设置meta key然后逆序查找
      // 逆序找方便，meta中使用的是table_regionStartRow_time作为rowKey
      // 逆序查找，根据start row定位后，next一下就能找到
      s.setStartRow(metaKey);
      s.setSmall(true);
      s.setCaching(1);
      if (this.useMetaReplicas) {
        s.setConsistency(Consistency.TIMELINE);
      }

      int localNumRetries = (retry ? numTries : 1);

      for (int tries = 0; true; tries++) {
        ...
        if (useCache) {
          RegionLocations locations = getCachedLocation(tableName, row);
          if (locations != null && locations.getRegionLocation(replicaId) != null) {
            return locations;
          }
        } else {
          // If we are not supposed to be using the cache, delete any existing cached location
          // so it won't interfere.
          metaCache.clearCache(tableName, row);
        }

        // Query the meta region
        try {
          Result regionInfoRow = null;
          ReversedClientScanner rcs = null;
          try {
            // scanner 内部会对meta表进行 regionLocation，scan和get，put不同，scan也不知道自己要访问多少region？所以应该是在内部创建rpc的时候再进行定位
            rcs = new ClientSmallReversedScanner(conf, s, TableName.META_TABLE_NAME, this,
              rpcCallerFactory, rpcControllerFactory, getMetaLookupPool(), 0);
            regionInfoRow = rcs.next();
          } finally {
            ...
          }
          ...
          // convert the row result into the HRegionLocation we need!
          RegionLocations locations = MetaTableAccessor.getRegionLocations(regionInfoRow);
          
          // Instantiate the location
          cacheLocation(tableName, locations);
          return locations;
        } 
        ...
      }
    }