Hbase-put调优

问题：

数据put一直报 table=daas:unicom_stat, attempt=11/35 failed=8ops, last exception: org.apache.hadoop.hbase.RegionTooBusyException: org.apache.hadoop.hbase.RegionTooBusyException: StoreTooBusy,daas:unicom_stat,1_47EA07A87638CD2F_2019-02-09_02,1555906135085.0cb7e9e40084f6118f21037379dcda96.:item Above parallelPutToStoreThreadLimit(10) ，

解析一下任务报错信息： 表daas:unicom_stat，region 1_47EA07A87638CD2F_2019-02-09_02,1555906135085.0cb7e9e40084f6118f21037379dcda96.且cf是item的RegionTooBusyException异常

然后我们来看源码

for (Map.Entry<byte[], List<Cell>> e : familyMaps.entrySet()) {
      Store store = this.region.getStore(e.getKey());
      if (store == null || e.getValue() == null) {
        continue;
      }
     // 如果当前cell的列大小超过默认值(100列)，则继续
      if (e.getValue().size() > this.parallelPutToStoreThreadLimitCheckMinColumnCount) {
​
        //we need to try to add #preparePutCount at first because preparePutToStoreMap will be
        //cleared when changing the configuration.
        preparePutToStoreMap.putIfAbsent(e.getKey(), new AtomicInteger());
        AtomicInteger preparePutCounter = preparePutToStoreMap.get(e.getKey());
        if (preparePutCounter == null) {
          preparePutCounter = new AtomicInteger();
          preparePutToStoreMap.putIfAbsent(e.getKey(), preparePutCounter);
        }
        int preparePutCount = preparePutCounter.incrementAndGet();
        // 如果当前region的某个storefile的并行put线程数超过10个，则进行异常捕获前的变量赋值，即获取regionname，cf等信息
        if (store.getCurrentParallelPutCount() > this.parallelPutToStoreThreadLimit
            || preparePutCount > this.parallelPreparePutToStoreThreadLimit) {
          tooBusyStore = (tooBusyStore == null ?
              store.getColumnFamilyName() :
              tooBusyStore + "," + store.getColumnFamilyName());
        }
​
        if (LOG.isTraceEnabled()) {
          LOG.trace(store.getColumnFamilyName() + ": preparePutCount=" + preparePutCount
              + "; currentParallelPutCount=" + store.getCurrentParallelPutCount());
        }
      }
    }
    //tooBusyStore此变量不为空则抛异常
    if (tooBusyStore != null) {
      String msg =
          "StoreTooBusy," + this.region.getRegionInfo().getRegionNameAsString() + ":" + tooBusyStore
              + " Above parallelPutToStoreThreadLimit(" + this.parallelPutToStoreThreadLimit + ")";
      if (LOG.isTraceEnabled()) {
        LOG.trace(msg);
      }
      throw new RegionTooBusyException(msg);
    }

所以很明显是由列数过多或者并行put线程数过多而引起的，这里是由于表字段太多，设置一下put时的列数>源表的列数即可（源表有200字段）

转载于:https://my.oschina.net/lzhaoqiang/blog/3064067