系列文章总目录
从github clone 最新源码,结构如下:
本篇文章要讲的通信与协议部分的源代码在remoting模块下。remoting模块是复杂网络通信的模块,为其他需要网络通信的模块所依赖。在这个模块中,RocketMQ定义了基础的通信协议,结合Netty,使得端与端之间的数据交互变得统一而高效。
基本类图
先来看这个模块的类关系图:
针对每个类分别做解释:
- RemotingService 为最上层接口
- RemotingClient 继承自RemotingService,提供了client端接口定义
- RemotingServer 继承自RemotingService,提供了server端接口定义
- NettyRemotingAbstract 使用Netty作为通信框架的抽象类,包含很多公共的处理逻辑和数据结构
- NettyRemotingClient 继承了NettyRemotingAbstract,并实现了RemotingClient接口,作为通信的client端
- NettyRemotingServer 继承了NettyRemotingAbstract,并实现了RemotingServer接口,作为通信的server端
- NettyEvent、NettyEncoder、NettyDecoder、RemotingCommand等为通信框架使用的类
通信协议
RocketMQ的通信协议如下:
整个通信消息分为四个部分:
- 整体消息长度,占用四个资格
- 序列化类型和消息头长度,一共占用四个字节,第一个字节表示序列化类型,后三个字节表示消息头长度
- 消息头数据
- 消息主体数据
源码详细分析之通信流程
RemotingService
public interface RemotingService {
void start();
void shutdown();
void registerRPCHook(RPCHook rpcHook);
}
RemotingService定义了三个基本的方法,registerRPCHook()注册一些钩子,用来做一些通信前后的处理。
RemotingClient 和 RemotingServer
public interface RemotingClient extends RemotingService {
void updateNameServerAddressList(final List<String> addrs);
List<String> getNameServerAddressList();
// 同步调用
RemotingCommand invokeSync(final String addr, final RemotingCommand request,
final long timeoutMillis) throws InterruptedException, RemotingConnectException,
RemotingSendRequestException, RemotingTimeoutException;
// 异步调用
void invokeAsync(final String addr, final RemotingCommand request, final long timeoutMillis,
final InvokeCallback invokeCallback) throws InterruptedException, RemotingConnectException,
RemotingTooMuchRequestException, RemotingTimeoutException, RemotingSendRequestException;
// 单向调用
void invokeOneway(final String addr, final RemotingCommand request, final long timeoutMillis)
throws InterruptedException, RemotingConnectException, RemotingTooMuchRequestException,
RemotingTimeoutException, RemotingSendRequestException;
void registerProcessor(final int requestCode, final NettyRequestProcessor processor,
final ExecutorService executor);
void setCallbackExecutor(final ExecutorService callbackExecutor);
ExecutorService getCallbackExecutor();
boolean isChannelWritable(final String addr);
}
public interface RemotingServer extends RemotingService {
void registerProcessor(final int requestCode, final NettyRequestProcessor processor,
final ExecutorService executor);
void registerDefaultProcessor(final NettyRequestProcessor processor, final ExecutorService executor);
int localListenPort();
Pair<NettyRequestProcessor, ExecutorService> getProcessorPair(final int requestCode);
// 同步调用
RemotingCommand invokeSync(final Channel channel, final RemotingCommand request,
final long timeoutMillis) throws InterruptedException, RemotingSendRequestException,
RemotingTimeoutException;
// 异步调用
void invokeAsync(final Channel channel, final RemotingCommand request, final long timeoutMillis,
final InvokeCallback invokeCallback) throws InterruptedException,
RemotingTooMuchRequestException, RemotingTimeoutException, RemotingSendRequestException;
// 单向调用
void invokeOneway(final Channel channel, final RemotingCommand request, final long timeoutMillis)
throws InterruptedException, RemotingTooMuchRequestException, RemotingTimeoutException,
RemotingSendRequestException;
}
比较重要的三个方法:同步调用、异步调用和单向调用。
NettyRemotingAbstract 部分核心代码
先看NettyRemotingAbstract定义的属性和构造方法
public abstract class NettyRemotingAbstract {
protected final Semaphore semaphoreOneway; // 单向调用信号量,控制并发
protected final Semaphore semaphoreAsync; // 异步调用信号量,控制并发
protected final ConcurrentMap<Integer /* opaque */, ResponseFuture> responseTable =
new ConcurrentHashMap<Integer, ResponseFuture>(256); // 处理中的请求,opaque是唯一的请求id
protected final HashMap<Integer/* request code */, Pair<NettyRequestProcessor, ExecutorService>> processorTable = // requestCode对应的请求处理Processor和线程池
new HashMap<Integer, Pair<NettyRequestProcessor, ExecutorService>>(64);
protected Pair<NettyRequestProcessor, ExecutorService> defaultRequestProcessor; // 默认处理Processor
public NettyRemotingAbstract(final int permitsOneway, final int permitsAsync) {
this.semaphoreOneway = new Semaphore(permitsOneway, true);
this.semaphoreAsync = new Semaphore(permitsAsync, true);
}
上面是定义的属性,两个信号量用来控制单向调用和异步调用的并发量。responseTable维护处理中的请求,请求处理完成后会进行剔除,详见下面的scanResponseTable()方法。processorTable维护requestCode和处理线程的映射关系,处理请求时从processorTable根据requestCode查询出Pair,使用Pair进行处理。构造方法初始化两个信号量。Semaphore继承自AQS,这里不详细描述,可以参见AQS源码学习。
下面是处理请求的逻辑, processMessageReceived()为入口,根据请求类型(request or reponse)做不同处理
public void processMessageReceived(ChannelHandlerContext ctx, RemotingCommand msg) throws Exception {
final RemotingCommand cmd = msg;
if (cmd != null) {
switch (cmd.getType()) {
case REQUEST_COMMAND:
processRequestCommand(ctx, cmd);
break;
case RESPONSE_COMMAND:
processResponseCommand(ctx, cmd);
break;
default:
break;
}
}
}
先来看处理Request的逻辑,使用processorTable中查询的processor和线程池进行逻辑处理,并进行rpcHook的调用。
public void processRequestCommand(final ChannelHandlerContext ctx, final RemotingCommand cmd) {
final Pair<NettyRequestProcessor, ExecutorService> matched = this.processorTable.get(cmd.getCode());// 根据请求类型查询处理的processor
final Pair<NettyRequestProcessor, ExecutorService> pair = null == matched ? this.defaultRequestProcessor : matched;
final int opaque = cmd.getOpaque();
if (pair != null) {
Runnable run = new Runnable() {
@Override
public void run() {
try {
RPCHook rpcHook = NettyRemotingAbstract.this.getRPCHook();
if (rpcHook != null) {
// 请求前置处理
rpcHook.doBeforeRequest(RemotingHelper.parseChannelRemoteAddr(ctx.channel()), cmd);
}
final RemotingCommand response = pair.getObject1().processRequest(ctx, cmd);
if (rpcHook != null) {
// 请求后置处理
rpcHook.doAfterResponse(RemotingHelper.parseChannelRemoteAddr(ctx.channel()), cmd, response);
}
if (!cmd.isOnewayRPC()) {
// 非单向请求,回写处理结果
if (response != null) {
response.setOpaque(opaque);
response.markResponseType();
try {
ctx.writeAndFlush(response);
} catch (Throwable e) {
// 略
}
} else {
}
}
} catch (Throwable e) {
// 略
}
}
};
if (pair.getObject1().rejectRequest()) {
//拒绝请求
final RemotingCommand response