HBase源代码调试(4)

protected class Connection extends Thread 

今天学习了一下RPC架构

When new a HTable object, it will create a special ServerName

ServerName("HBCKServerName", -1, -1)
///** HBCK special code name used as server name when manipulating ZK nodes */

specify hard code Non-User Dir
[.logs, .oldlogs, .corrupt, .META., -ROOT-, splitlog, .hbck] 

-----------------------------------------------------------------------------

In HConnectionManager.java 
function 

HRegionInterface getHRegionConnection(final String hostname, final int port,
        final InetSocketAddress isa, final boolean master)

server = (HRegionInterface) HBaseRPC.waitForProxy(
                  serverInterfaceClass, HRegionInterface.VERSION,
                  address, this.conf,
                  this.maxRPCAttempts, this.rpcTimeout, this.rpcTimeout);

In HBaseRPC.java

 public static VersionedProtocol getProxy(
      Class<? extends VersionedProtocol> protocol,
      long clientVersion, InetSocketAddress addr, User ticket,
      Configuration conf, SocketFactory factory, int rpcTimeout)
  throws IOException {
    VersionedProtocol proxy =
        getProtocolEngine(protocol,conf)
            .getProxy(protocol, clientVersion, addr, ticket, conf, factory, Math.min(rpcTimeout, HBaseRPC.getRpcTimeout()));
    long serverVersion = proxy.getProtocolVersion(protocol.getName(),
                                                  clientVersion);
    if (serverVersion == clientVersion) {
      return proxy;
    }
    throw new VersionMismatch(protocol.getName(), clientVersion,
                              serverVersion);
  }

getProtocolEngine   will return a  WritableRpcEngine object

In WritableRpcEngine.java

  public VersionedProtocol getProxy(
      Class<? extends VersionedProtocol> protocol, long clientVersion,
      InetSocketAddress addr, User ticket,
      Configuration conf, SocketFactory factory, int rpcTimeout)
    throws IOException {

      VersionedProtocol proxy =
          (VersionedProtocol) Proxy.newProxyInstance(
              protocol.getClassLoader(), new Class[] { protocol },
              new Invoker(protocol, addr, ticket, conf, factory, rpcTimeout));
    if (proxy instanceof VersionedProtocol) {
      long serverVersion = ((VersionedProtocol)proxy)
        .getProtocolVersion(protocol.getName(), clientVersion);
      if (serverVersion != clientVersion) {
        throw new HBaseRPC.VersionMismatch(protocol.getName(), clientVersion,
                                      serverVersion);
      }
    }
    return proxy;
  }

其中Invoker是它的内嵌static类

private static class Invoker implements InvocationHandler {
    private Class<? extends VersionedProtocol> protocol;
    private InetSocketAddress address;
    private User ticket;
    private HBaseClient client;
    private boolean isClosed = false;
    final private int rpcTimeout;

    public Invoker(Class<? extends VersionedProtocol> protocol,
                   InetSocketAddress address, User ticket,
                   Configuration conf, SocketFactory factory, int rpcTimeout) {
      this.protocol = protocol;
      this.address = address;
      this.ticket = ticket;
      this.client = CLIENTS.getClient(conf, factory);
      this.rpcTimeout = rpcTimeout;
    }

    public Object invoke(Object proxy, Method method, Object[] args)
        throws Throwable {
      final boolean logDebug = LOG.isDebugEnabled();
      long startTime = 0;
      if (logDebug) {
        startTime = System.currentTimeMillis();
      }

      HbaseObjectWritable value = (HbaseObjectWritable)
        client.call(new Invocation(method, protocol, args), address,
                    protocol, ticket, rpcTimeout);
      if (logDebug) {
        // FIGURE HOW TO TURN THIS OFF!
        long callTime = System.currentTimeMillis() - startTime;
        LOG.debug("Call: " + method.getName() + " " + callTime);
      }
      return value.get();
    }

    /* close the IPC client that's responsible for this invoker's RPCs */
    synchronized protected void close() {
      if (!isClosed) {
        isClosed = true;
        CLIENTS.stopClient(client);
      }
    }
  }

protocol is org.apache.hadoop.hbase.ipc.HRegionInterface

set in configuration file   "hbase.regionserver.class"  default value is "HConstants.DEFAULT_REGION_SERVER_CLASS"

最后都会调用到Invoker.invoke,   然后到client.call ()

in HBaseClient.java

public Writable call(Writable param, InetSocketAddress addr,
                       Class<? extends VersionedProtocol> protocol,
                       User ticket, int rpcTimeout)
      throws InterruptedException, IOException {
    Call call = new Call(param);
    Connection connection = getConnection(addr, protocol, ticket, rpcTimeout, call);
    connection.sendParam(call); 

  protected class Call {
    final int id;                                       // call id
    final Writable param;                               // parameter
    Writable value;                               // value, null if error
    IOException error;                            // exception, null if value
    boolean done;                                 // true when call is done
    long startTime;

    protected Call(Writable param) {
      this.param = param;
      this.startTime = System.currentTimeMillis();
      synchronized (HBaseClient.this) {
        this.id = counter++;
      }
    }

//Call 又是一个内部类

所以当我们在

HConnectionManager.java里面调用如

regionInfoRow = server.getClosestRowBefore(
            metaLocation.getRegionInfo().getRegionName(), metaKey,
            HConstants.CATALOG_FAMILY);

我们并非在client端调用某个类的getClosestRowBefore, 而是把"getClosestRowBefore"这个方法的名字等信息

封装成一个Invocation object, 进而传递给Call 类的构造函数, connection.sendParam(call)发送一个请求.

在此之前

Connection connection=getConnection(addr, ...)会启动一个线程来接受服务器的response

Connection是一个HBaseClient的内部类

protected class Connection extends Thread 

-------------------------------------------------------------------------------------------------------------------------------------

in processBatcchCallback function
   public <R> void processBatchCallback(
        List<? extends Row> list,
        byte[] tableName,
        ExecutorService pool,
        Object[] results,
        Batch.Callback<R> callback)
    throws IOException, InterruptedException{
       for (Entry<HRegionLocation, MultiAction<R>> e: actionsByServer.entrySet()) {
          futures.put(e.getKey(), pool.submit(createCallable(e.getKey(), e.getValue(), tableName)));
        }

        // step 3: collect the failures and successes and prepare for retry

        for (Entry<HRegionLocation, Future<MultiResponse>> responsePerServer
             : futures.entrySet()) {
          HRegionLocation loc = responsePerServer.getKey();

          try {
            Future<MultiResponse> future = responsePerServer.getValue();
            MultiResponse resp = future.get();


let's look at createCallable function
   private <R> Callable<MultiResponse> createCallable(final HRegionLocation loc,
        final MultiAction<R> multi, final byte [] tableName) {
      // TODO: This does not belong in here!!! St.Ack  HConnections should
      // not be dealing in Callables; Callables have HConnections, not other
      // way around.
      final HConnection connection = this;
      return new Callable<MultiResponse>() {
       public MultiResponse call() throws IOException {
         ServerCallable<MultiResponse> callable =
           new ServerCallable<MultiResponse>(connection, tableName, null) {
             public MultiResponse call() throws IOException {
               return server.multi(multi);
             }
             @Override
             public void connect(boolean reload) throws IOException {
               server = connection.getHRegionConnection(loc.getHostname(), loc.getPort());
             }
           };
         return callable.withoutRetries();
       }
     };
   }

When we call  
MultiResponse resp = future.get();
it will call 
callable.withoutRetries();
The definition for withoutRetries
 public T withoutRetries()
  throws IOException, RuntimeException {
    try {
      beforeCall();
      connect(false);
      return call();
    } catch (Throwable t) {
      Throwable t2 = translateException(t);
      if (t2 instanceof IOException) {
        throw (IOException)t2;
      } else {
        throw new RuntimeException(t2);
      }
    } finally {
      afterCall();
    }
  }


then call its definition for connect(false)  and call()
            public MultiResponse call() throws IOException {
               return server.multi(multi);
             }
             @Override
             public void connect(boolean reload) throws IOException {
               server = connection.getHRegionConnection(loc.getHostname(), loc.getPort());
             }

--------------------------------------------------------------------

In WritableRpcEngine.java
Construct a HBaseClient object

protected synchronized HBaseClient getClient(Configuration conf,
        SocketFactory factory) {
      // Construct & cache client.  The configuration is only used for timeout,
      // and Clients have connection pools.  So we can either (a) lose some
      // connection pooling and leak sockets, or (b) use the same timeout for
      // all configurations.  Since the IPC is usually intended globally, not
      // per-job, we choose (a).
      HBaseClient client = clients.get(factory);
      if (client == null) {
        // Make an hbase client instead of hadoop Client.
        client = new HBaseClient(HbaseObjectWritable.class, conf, factory);
        clients.put(factory, client);
      } else {
        client.incCount();
      }
      return client;
    }

HbaseObjectWritable will be the 'valueClass' of object

so In 
protected void receiveResponse() {   //in HBaseClient$Connection
	Writable value = ReflectionUtils.newInstance(valueClass, conf); //the 'value' will be a new object of Class HbaseObjectWritable


also in Connection, there is a var 'calls', it contains all the request to the server, and the return value will also be writen to calls (identified by id)
 // currently active calls
    protected final ConcurrentSkipListMap<Integer, Call> calls = new ConcurrentSkipListMap<Integer, Call>();


  public Writable call(Writable param, InetSocketAddress addr,
                       Class<? extends VersionedProtocol> protocol,
                       User ticket, int rpcTimeout)
      throws InterruptedException, IOException {
    Call call = new Call(param);
    Connection connection = getConnection(addr, protocol, ticket, rpcTimeout, call);//Add request 'call' to 'calls' and start a thread to get response by function 'receiveResponse', the return value be written to new HbaseObjectWritable object, and assign to call 
    connection.sendParam(call);   //sent request to server?

-------------------------------
HRegion
可以单独调试学习, 只要在debug configurations里面
main class 指定为HRegion 
arguments 指定一个catalog table
The so called Catalog table is '.META.' & '-ROOT-'

when start up , read default block size through

  public long getDefaultBlockSize() {
    // default to 32MB: large enough to minimize the impact of seeks
    return getConf().getLong("fs.local.block.size", 32 * 1024 * 1024);
  }