TCP/IP协议在通信过程中会出现粘包和拆包的现象,
用一张图来说明正常情况是D1 D2数据如第一行所示,而发生了粘包就如同第二行图所示,拆包这是第三。四两张图所示
一旦发生粘包和拆包,上层的协议就无法理解
由于底层的TCP无法理解上层的业务数据,所以在底层是无法保证数据包不被拆分和重组的,这个问题只能通过上层的应用协议栈设计来解决,根据业界的主流协议的解决方案,可以归纳如下。
(1)消息定长,例如每个报文的大小为固定长度200字节,如果不够,空位补空格;
(2)在包尾增加回车换行符进行分割,例如FTP协议;
(3)将消息分为消息头和消息体,消息头中包含表示消息总长度(或者消息体长度)的字段,通常设计思路为消息头的第一个字段使用int32来表示消息的总长度;
(4)更复杂的应用层协议。
dubbo在TCP/IP协议的上层封装了一层dubbo协议,采用了(3)号方式来解决粘包和拆包的问题
附上一张官网的图
dubbo协议采用固定长度的消息头(16字节)和不定长度的消息体来进行数据传输
消息头详解
协议头是16字节的定长数据:
2byte magic:类似java字节码文件里的魔数,用来判断是不是dubbo协议的数据包。魔数是常量0xdabb
1byte 的消息标志位:16-20序列id,21 event,22 two way,23请求或响应标识
1byte 状态,当消息类型为响应时,设置响应状态。24-31位。状态位, 设置请求响应状态,dubbo定义了一些响应的类型。具体类型见com.alibaba.dubbo.remoting.exchange.Response
8byte 消息ID,long类型,32-95位。每一个请求的唯一识别id(由于采用异步通讯的方式,用来把请求request和返回的response对应上)
4byte 消息长度,96-127位。消息体 body 长度, int 类型,即记录Body Content有多少个字节。
直接进入ExchangeCodec#encodeRequest
byte[] header = new byte[HEADER_LENGTH];这里设置了一个定长的消息头
Bytes.short2bytes(MAGIC, header)
然后设置了一个魔数 占header[0]和header[1]
if (req.isTwoWay()) header[2] |= FLAG_TWOWAY; if (req.isEvent()) header[2] |= FLAG_EVENT;
这里根据请求是双向还是单向设置一个标识 占header[2]
Bytes.long2bytes(req.getId(), header, 4); 占header[4]-heander[11]
int savedWriteIndex = buffer.writerIndex(); buffer.writerIndex(savedWriteIndex + HEADER_LENGTH);
获取buffer的写入位置writerIndex, 加上消息头长度16,重新设置buffer的写入位置,这里是消息body的写入位置, 因为后面是先写body,要把header的位置空出来
ChannelBufferOutputStream bos = new ChannelBufferOutputStream(buffer); ObjectOutput out = serialization.serialize(channel.getUrl(), bos); if (req.isEvent()) { encodeEventData(channel, out, req.getData()); } else { encodeRequestData(channel, out, req.getData()); }
序列化消息body, (request, response参考前面的)写入buffer
DubboCodec#encodeRequestData
@Override protected void encodeRequestData(Channel channel, ObjectOutput out, Object data) throws IOException { RpcInvocation inv = (RpcInvocation) data; out.writeUTF(inv.getAttachment(Constants.DUBBO_VERSION_KEY, DUBBO_VERSION));//版本号 out.writeUTF(inv.getAttachment(Constants.PATH_KEY));//invoke的路径 out.writeUTF(inv.getAttachment(Constants.VERSION_KEY));//invoke的provider端暴露的服务的版本号 out.writeUTF(inv.getMethodName());//调用的方法名称 out.writeUTF(ReflectUtils.getDesc(inv.getParameterTypes()));//参数类型描述符 Object[] args = inv.getArguments(); if (args != null) for (int i = 0; i < args.length; i++){ out.writeObject(encodeInvocationArgument(channel, inv, i));//遍历请求参数值并编码 } out.writeObject(inv.getAttachments());//dubbo请求的attachments }
int len = bos.writtenBytes();//计算消息体大小writerIndex – startIndex checkPayload(channel, len);//检查消息体是否超过限制大小 Bytes.int2bytes(len, header, 12);//给消息头数组最后四位写入消息消body长度值int类型 这里占据Header[12]-header[15] // write buffer.writerIndex(savedWriteIndex);//消息头数组最后四位写入消息消body长度值int类型 buffer.writeBytes(header); // buffer写入消息头数据 buffer.writerIndex(savedWriteIndex + HEADER_LENGTH + len);//Buffer设置writerIndex=savedWriteIndex+ HEADER_LENGTH + len
------------------------------------------------------------------------------------------------
以上部分是client端的编码,接下去看server端的解码
ExchangeCodec#decode方法
public Object decode(Channel channel, ChannelBuffer buffer) throws IOException { int readable = buffer.readableBytes();//从channle获取可读取的字节数readable byte[] header = new byte[Math.min(readable, HEADER_LENGTH)]; //readable跟header_length取小构建字节数组header[] buffer.readBytes(header);// return decode(channel, buffer, readable, header);//开始解码 }
protected Object decode(Channel channel, ChannelBuffer buffer, int readable, byte[] header) throws IOException { 这里会判断header[]的第一个和第二个字节不是dubbo协议魔数 if (readable > 0 && header[0] != MAGIC_HIGH || readable > 1 && header[1] != MAGIC_LOW) { 如果可读取字节readable大于header_length, 重新构建header[], 读取全部可读取readable数据到header int length = header.length; if (header.length < readable) { header = Bytes.copyOf(header, readable); buffer.readBytes(header, length, readable - length); }
遍历header中的字节判断中间是否有dubbo协议的魔数0xdabb, 如果有说明魔数开始的是dubbo协议的消息。
重新设置buffer的读取位置从魔数开始
截取header[]中从头开始到魔术位置的数据
for (int i = 1; i < header.length - 1; i ++) { if (header[i] == MAGIC_HIGH && header[i + 1] == MAGIC_LOW) { buffer.readerIndex(buffer.readerIndex() - header.length + i); header = Bytes.copyOf(header, i); break; } }
return super.decode(channel, buffer, readable, header); } 。如果readable < header_length说明不是一个全的dubbo协议消息(所以后面要判断消息头魔数),或者只是一个telnet消息 从返回的枚举也能看出是消息头长度不够 if (readable < HEADER_LENGTH) { return DecodeResult.NEED_MORE_INPUT; } // get data length. int len = Bytes.bytes2int(header, 12); 读取header[]最后四位一个int的数据,bodydata的长度len checkPayload(channel, len); int tt = len + HEADER_LENGTH; 计算总消息大小tt = len + body_length 如果可读取数据readable < tt, 数据不够返回NEED_MORE_INPUT if( readable < tt ) { return DecodeResult.NEED_MORE_INPUT; }
由buffer和len构建ChannelBufferInputStream, 后面序列化工具会使用 ChannelBufferInputStream is = new ChannelBufferInputStream(buffer, len);
下面是解码消息体body data过程 try { return decodeBody(channel, is, header);
进入
DubboDedec #decode
// decode request. Request req = new Request(id); req.setVersion("2.0.0"); req.setTwoWay((flag & FLAG_TWOWAY) != 0); if ((flag & FLAG_EVENT) != 0) { req.setEvent(Request.HEARTBEAT_EVENT); } try { Object data; if (req.isHeartbeat()) { data = decodeHeartbeatData(channel, deserialize(s, channel.getUrl(), is)); } else if (req.isEvent()) { data = decodeEventData(channel, deserialize(s, channel.getUrl(), is)); } else { DecodeableRpcInvocation inv; if (channel.getUrl().getParameter( Constants.DECODE_IN_IO_THREAD_KEY, Constants.DEFAULT_DECODE_IN_IO_THREAD)) { inv = new DecodeableRpcInvocation(channel, req, is, proto); inv.decode(); } else { inv = new DecodeableRpcInvocation(channel, req, new UnsafeByteArrayInputStream(readMessageData(is)), proto); } data = inv; } req.setData(data);
首先new 一个requset,并且把id设置上去。设置版本号之类的
然后回new一个DecodeableRpcInvocation
其构造器是
this.channel = channel; this.request = request; this.inputStream = is; this.serializationType = id;
这个就是传输的数据
-----------------------------------------------------------
再看provider端是如何编码返回的result的
ExchangerCodec#encodeResponse
大部分代码和encodeRequest是一样的,只看部分不一样的代码
Serialization serialization = getSerialization(channel);首先获取序列化方式
header[2] = serialization.getContentTypeId(); header[2]存储序列化方式的id。默认是hession2
header[3] = status; header[3]存储了response的状态
然后进入
encodeResponseData(channel, out, res.getResult());
DubboCodec#encodeResponseData @Override protected void encodeResponseData(Channel channel, ObjectOutput out, Object data) throws IOException { Result result = (Result) data; Throwable th = result.getException(); if (th == null) { Object ret = result.getValue();//获取结果 if (ret == null) { out.writeByte(RESPONSE_NULL_VALUE); //如果结果为空 则写入2 } else { out.writeByte(RESPONSE_VALUE);//结果不为空就写入1 out.writeObject(ret);并且写入结果 } } else { out.writeByte(RESPONSE_WITH_EXCEPTION);//抛异常 out.writeObject(th);//异常写入 } }
剩下的都与consumer端编码一样
-------------------------------------------------
最后看consumer端怎么解码返回的result
前面一部分都和provider端处理request一样
DubboCodec#decodeBody
Response res = new Response(id); if ((flag & FLAG_EVENT) != 0) { res.setEvent(Response.HEARTBEAT_EVENT); } // get status. byte status = header[3]; res.setStatus(status); if (status == Response.OK) { try { Object data; if (res.isHeartbeat()) { data = decodeHeartbeatData(channel, deserialize(s, channel.getUrl(), is)); } else if (res.isEvent()) { data = decodeEventData(channel, deserialize(s, channel.getUrl(), is)); } else { DecodeableRpcResult result; if (channel.getUrl().getParameter( Constants.DECODE_IN_IO_THREAD_KEY, Constants.DEFAULT_DECODE_IN_IO_THREAD)) { result = new DecodeableRpcResult(channel, res, is, (Invocation)getRequestData(id), proto); result.decode(); } else { result = new DecodeableRpcResult(channel, res, new UnsafeByteArrayInputStream(readMessageData(is)), (Invocation) getRequestData(id), proto); } data = result; } res.setResult(data);
看这里
byte status = header[3];会获取response的状态
如果状态为20就会写入结果
DecodeableRpcResult result; if (channel.getUrl().getParameter( Constants.DECODE_IN_IO_THREAD_KEY, Constants.DEFAULT_DECODE_IN_IO_THREAD)) { result = new DecodeableRpcResult(channel, res, is, (Invocation)getRequestData(id), proto); result.decode(); } else { result = new DecodeableRpcResult(channel, res, new UnsafeByteArrayInputStream(readMessageData(is)), (Invocation) getRequestData(id), proto); } data = result;
会new 一个DecodeableRpcResult
this.channel = channel; this.response = response; this.inputStream = is; this.invocation = invocation; this.serializationType = id;
构造函数中会初始化response,序列化id入参等
result.decode();点进入
byte flag = in.readByte(); switch (flag) { case DubboCodec.RESPONSE_NULL_VALUE: break; case DubboCodec.RESPONSE_VALUE: try { Type[] returnType = RpcUtils.getReturnTypes(invocation); setValue(returnType == null || returnType.length == 0 ? in.readObject() : (returnType.length == 1 ? in.readObject((Class<?>) returnType[0]) : in.readObject((Class<?>) returnType[0], returnType[1]))); } catch (ClassNotFoundException e) { throw new IOException(StringUtils.toString("Read response data failed.", e)); } break; case DubboCodec.RESPONSE_WITH_EXCEPTION: try { Object obj = in.readObject(); if (obj instanceof Throwable == false) throw new IOException("Response data error, expect Throwable, but get " + obj); setException((Throwable) obj); } catch (ClassNotFoundException e) { throw new IOException(StringUtils.toString("Read response data failed.", e)); } break; default:
会根据flag来执行不同判断。
看DubboCodec.RESPONSE_VALUE:有值情况
Type[] returnType = RpcUtils.getReturnTypes(invocation); setValue(returnType == null || returnType.length == 0 ? in.readObject() : (returnType.length == 1 ? in.readObject((Class<?>) returnType[0]) : in.readObject((Class<?>) returnType[0], returnType[1])));会设置上值
最后回到ExchangerCodec#decode,方法调用完之后会回到
DubboCountCodec#decode
do { Object obj = codec.decode(channel, buffer); if (Codec2.DecodeResult.NEED_MORE_INPUT == obj) { buffer.readerIndex(save); break; } else { result.addMessage(obj); logMessageLength(obj, buffer.readerIndex() - save); save = buffer.readerIndex(); } } while (true);
这个死循环中就是直到读取到的数据满足消息头是16个字节为止。
--------------------------------
dubbo的源码分析系列结束。为了配合dubbo,接下去将会分析netty的源码