HBase 1.2.5 put 源码分析

1. HTable 调用 put(final Put put) 方法,本地有一个缓存，不是每一次put就会提交到服务器端.

  @Override
  public void put(final Put put) throws IOException {
    getBufferedMutator().mutate(put);
    if (autoFlush) {
      flushCommits();
    }
  }

2. 继续深入，发现拿到了一个管理缓存的类. BufferedMutatorImpl，可以看到这个类里面有本地缓存数据的大小

  @VisibleForTesting
  BufferedMutator getBufferedMutator() throws IOException {
    if (mutator == null) {
      this.mutator = (BufferedMutatorImpl) connection.getBufferedMutator(
          new BufferedMutatorParams(tableName)
              .pool(pool)
              .writeBufferSize(connConfiguration.getWriteBufferSize())
              .maxKeyValueSize(connConfiguration.getMaxKeyValueSize())
      );
    }
    return mutator;
  }
}

3. 接下来看这个缓存类怎么将put 缓存着，还有什么时候提交给服务器端.即看 mutate(put) 方法.

    a. Put 是Mutation的派生类，调用 mutate(List<? extends Mutation> ms)

    b. 将List<? extends Mutation> ms 加入到本地的 

final ConcurrentLinkedQueue<Mutation> writeAsyncBuffer = new ConcurrentLinkedQueue<Mutation>(); 中，如果 缓存值大于配置的值，就会调用 调用backgroundFlushCommits(bool synchronous) 发送到服务器端.

@Override
  public void mutate(Mutation m) throws InterruptedIOException,
      RetriesExhaustedWithDetailsException {
    mutate(Arrays.asList(m));
  }

  @Override
  public void mutate(List<? extends Mutation> ms) throws InterruptedIOException,
      RetriesExhaustedWithDetailsException {

    if (closed) {
      throw new IllegalStateException("Cannot put when the BufferedMutator is closed.");
    }

    long toAddSize = 0;
    for (Mutation m : ms) {
      if (m instanceof Put) {
        validatePut((Put) m);
      }
      toAddSize += m.heapSize();
    }

    // This behavior is highly non-intuitive... it does not protect us against
    // 94-incompatible behavior, which is a timing issue because hasError, the below code
    // and setter of hasError are not synchronized. Perhaps it should be removed.
    if (ap.hasError()) {
      currentWriteBufferSize.addAndGet(toAddSize);
      writeAsyncBuffer.addAll(ms);
      backgroundFlushCommits(true);
    } else {
      currentWriteBufferSize.addAndGet(toAddSize);
      writeAsyncBuffer.addAll(ms);
    }

    // Now try and queue what needs to be queued.
    while (currentWriteBufferSize.get() > writeBufferSize) {
      backgroundFlushCommits(false);
    }
  }

4.  做一些预处理，然后调用 protected AsyncProcess ap; 的 submit 方法

 private void backgroundFlushCommits(boolean synchronous) throws
      InterruptedIOException,
      RetriesExhaustedWithDetailsException {

    LinkedList<Mutation> buffer = new LinkedList<>();
    // Keep track of the size so that this thread doesn't spin forever
    long dequeuedSize = 0;

    try {
      // Grab all of the available mutations.
      Mutation m;

      // If there's no buffer size drain everything. If there is a buffersize drain up to twice
      // that amount. This should keep the loop from continually spinning if there are threads
      // that keep adding more data to the buffer.
      while (
          (writeBufferSize <= 0 || dequeuedSize < (writeBufferSize * 2) || synchronous)
              && (m = writeAsyncBuffer.poll()) != null) {
        // buffer 是即将提交到服务器端的put数据
        buffer.add(m);
        long size = m.heapSize();
        dequeuedSize += size;
        currentWriteBufferSize.addAndGet(-size);
      }

      if (!synchronous && dequeuedSize == 0) {
        return;
      }

      if (!synchronous) {
        ap.submit(tableName, buffer, true, null, false);
        if (ap.hasError()) {
          LOG.debug(tableName + ": One or more of the operations have failed -"
              + " waiting for all operation in progress to finish (successfully or not)");
        }
      }
      if (synchronous || ap.hasError()) {
        while (!buffer.isEmpty()) {
          //调用 protected AsyncProcess ap; 的sbumit 方法
          ap.submit(tableName, buffer, true, null, false);
        }
        RetriesExhaustedWithDetailsException error =
            ap.waitForAllPreviousOpsAndReset(null, tableName.getNameAsString());
        if (error != null) {
          if (listener == null) {
            throw error;
          } else {
            this.listener.onException(error, this);
          }
        }
      }
    } finally {
      for (Mutation mut : buffer) {
        long size = mut.heapSize();
        currentWriteBufferSize.addAndGet(size);
        dequeuedSize -= size;
        writeAsyncBuffer.add(mut);
      }
    }
  }

5. 将put 按照regionserver分类，然后 调用 submitMultiActions 进行提交.

  public <CResult> AsyncRequestFuture submit(ExecutorService pool, TableName tableName,
      List<? extends Row> rows, boolean atLeastOne, Batch.Callback<CResult> callback,
      boolean needResults) throws InterruptedIOException {
    if (rows.isEmpty()) {
      return NO_REQS_RESULT;
    }

    Map<ServerName, MultiAction<Row>> actionsByServer =
        new HashMap<ServerName, MultiAction<Row>>();
    List<Action<Row>> retainedActions = new ArrayList<Action<Row>>(rows.size());

    NonceGenerator ng = this.connection.getNonceGenerator();
    long nonceGroup = ng.getNonceGroup(); // Currently, nonce group is per entire client.

    // Location errors that happen before we decide what requests to take.
    List<Exception> locationErrors = null;
    List<Integer> locationErrorRows = null;
    do {
      // Wait until there is at least one slot for a new task.
      waitForMaximumCurrentTasks(maxTotalConcurrentTasks - 1, tableName.getNameAsString());

      // Remember the previous decisions about regions or region servers we put in the
      //  final multi.
      Map<HRegionInfo, Boolean> regionIncluded = new HashMap<HRegionInfo, Boolean>();
      Map<ServerName, Boolean> serverIncluded = new HashMap<ServerName, Boolean>();

      int posInList = -1;
      Iterator<? extends Row> it = rows.iterator();
      while (it.hasNext()) {
        Row r = it.next();
        HRegionLocation loc;
        try {
          if (r == null) {
            throw new IllegalArgumentException("#" + id + ", row cannot be null");
          }
          // Make sure we get 0-s replica.
          RegionLocations locs = connection.locateRegion(
              tableName, r.getRow(), true, true, RegionReplicaUtil.DEFAULT_REPLICA_ID);
          if (locs == null || locs.isEmpty() || locs.getDefaultRegionLocation() == null) {
            throw new IOException("#" + id + ", no location found, aborting submit for"
                + " tableName=" + tableName + " rowkey=" + Bytes.toStringBinary(r.getRow()));
          }
          loc = locs.getDefaultRegionLocation();
        } catch (IOException ex) {
          locationErrors = new ArrayList<Exception>();
          locationErrorRows = new ArrayList<Integer>();
          LOG.error("Failed to get region location ", ex);
          // This action failed before creating ars. Retain it, but do not add to submit list.
          // We will then add it to ars in an already-failed state.
          retainedActions.add(new Action<Row>(r, ++posInList));
          locationErrors.add(ex);
          locationErrorRows.add(posInList);
          it.remove();
          break; // Backward compat: we stop considering actions on location error.
        }

        if (canTakeOperation(loc, regionIncluded, serverIncluded)) {
          Action<Row> action = new Action<Row>(r, ++posInList);
          setNonce(ng, r, action);
          retainedActions.add(action);
          // TODO: replica-get is not supported on this path
          byte[] regionName = loc.getRegionInfo().getRegionName();
          // 在这个方法中 根据server，将put分类，这样发送到同一个server的put会被一起处理
          addAction(loc.getServerName(), regionName, action, actionsByServer, nonceGroup);
          it.remove();
        }
      }
    } while (retainedActions.isEmpty() && atLeastOne && (locationErrors == null));

    if (retainedActions.isEmpty()) return NO_REQS_RESULT;

    return submitMultiActions(tableName, retainedActions, nonceGroup, callback, null, needResults,
      locationErrors, locationErrorRows, actionsByServer, pool);
  }

6.  submitMultiActions =>  sendMultiAction, 针对每一个 server 调用  getNewMultiActionRunnable（）生成一个处理的线程.即SingleServerRequestRunnable.,可以看到最后实际用的是 

callable = createCallable(server, tableName, multiAction);，继续跟下去会发现是调用的 MultiServerCallable 中的 call 函数中的 responseProto = getStub().multi(controller, requestProto); 将put 提交到 server 中.

 private final class SingleServerRequestRunnable implements Runnable {
      private final MultiAction<Row> multiAction;
      private final int numAttempt;
      private final ServerName server;
      private final Set<MultiServerCallable<Row>> callsInProgress;

      private SingleServerRequestRunnable(
          MultiAction<Row> multiAction, int numAttempt, ServerName server,
          Set<MultiServerCallable<Row>> callsInProgress) {
        this.multiAction = multiAction;
        this.numAttempt = numAttempt;
        this.server = server;
        this.callsInProgress = callsInProgress;
      }

      @Override
      public void run() {
        MultiResponse res;
        MultiServerCallable<Row> callable = null;
        try {
          callable = createCallable(server, tableName, multiAction);
          try {
            RpcRetryingCaller<MultiResponse> caller = createCaller(callable);
            if (callsInProgress != null) callsInProgress.add(callable);
            res = caller.callWithoutRetries(callable, timeout);

            if (res == null) {
              // Cancelled
              return;
            }