hbase源码分析-是否split

@Override
  protected boolean shouldSplit() {
    if (region.shouldForceSplit()) return true;
    boolean foundABigStore = false;
    // Get count of regions that have the same common table as this.region 获取到该regionserver下tablename下所有的region个数
    int tableRegionsCount = getCountOfCommonTableRegions();
    // Get size to check
    long sizeToCheck = getSizeToCheck(tableRegionsCount);

    for (Store store : region.getStores()) {
      // If any of the stores is unable to split (eg they contain reference files)
      // then don't split
      if ((!store.canSplit())) {
        return false;
      }

      // Mark if any store is big enough
      long size = store.getSize();
      if (size > sizeToCheck) {
        LOG.debug("ShouldSplit because " + store.getColumnFamilyName() +
          " size=" + size + ", sizeToCheck=" + sizeToCheck +
          ", regionsWithCommonTable=" + tableRegionsCount);
        foundABigStore = true;
      }
    }

    return foundABigStore;
  }

 /**
   * @return Count of regions on this server that share the table this.region
   * belongs to
   */
  private int getCountOfCommonTableRegions() {
    RegionServerServices rss = this.region.getRegionServerServices();
    // Can be null in tests
    if (rss == null) return 0;
    TableName tablename = this.region.getTableDesc().getTableName();
    int tableRegionsCount = 0;
    try {
      List<Region> hri = rss.getOnlineRegions(tablename);
      tableRegionsCount = hri == null || hri.isEmpty()? 0: hri.size();
    } catch (IOException e) {
      LOG.debug("Failed getOnlineRegions " + tablename, e);
    }
    return tableRegionsCount;
  }
</pre><pre>
 
 如果tableRegionsCount ==0或者tableRegionsCount 》100,sizeToCheck 取desiredMaxFileSize,否则取desiredMaxFileSize和this.initialSize * tableRegionsCount * tableRegionsCount * tableRegionsCount的最小值,initialSize取this.initialSize = conf.getLong("hbase.increasing.policy.initial.size", -1); 
采用默认,一张表刚建立,默认情况只有1个region,那么逻辑上是当这个region的store大小超过 1*1*1*flushsize = 128M(没有自己设置flushSize)时 才会允许split,如果达到这个值切分后,会有两个region,其中一个region中的某个store大小大于 2*2*2* 
flushsize =  
512M 时,则允许split,如此计算下去,直到这个大小超过了 
hbase.hregion.max.filesizehbase.hregion.max.filesize*随机小数* 
hbase.hregion.max.filesize.jitter才允许split,基本也就固定了,如果粗劣的计算可以把这个 
hbase.hregion.max.filesize 
的大小作为最后的阀值,默认是10G,也就说当这个阀值变化到10G,这个阀值就基本上不再变化。 
 /**
   * @return Region max size or <code>count of regions squared * flushsize, which ever is
   * smaller; guard against there being zero regions on this server.
   */
  protected long getSizeToCheck(final int tableRegionsCount) {
    // safety check for 100 to avoid numerical overflow in extreme cases
    return tableRegionsCount == 0 || tableRegionsCount > 100 ? getDesiredMaxFileSize():
      Math.min(getDesiredMaxFileSize(),
        this.initialSize * tableRegionsCount * tableRegionsCount * tableRegionsCount);
  }
<span style="font-family: Arial, Helvetica, sans-serif;">long getDesiredMaxFileSize() {</span>
    return desiredMaxFileSize;
  }

@Override
  protected void configureForRegion(HRegion region) {
    super.configureForRegion(region);
    Configuration conf = getConf();
    HTableDescriptor desc = region.getTableDesc();
    if (desc != null) {
      this.desiredMaxFileSize = desc.getMaxFileSize();
    }
    if (this.desiredMaxFileSize <= 0) {
      this.desiredMaxFileSize = conf.getLong(HConstants.HREGION_MAX_FILESIZE,
        HConstants.DEFAULT_MAX_FILE_SIZE);
    }
    double jitter = conf.getDouble("hbase.hregion.max.filesize.jitter", 0.25D);
    this.desiredMaxFileSize += (long)(desiredMaxFileSize * (RANDOM.nextFloat() - 0.5D) * jitter);
  }
代码逻辑都比较简单,这里最这

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值