在之前的一篇《netty模拟TCP数据粘包》中提到过,若在同一条连接中发送的数据过快时,会发送数据粘包的情况,如下图:
前面博客提到过,解决这种问题,netty本身提供了三种解决方案:LineBasedFrameDecoder、DeLimiterBasedFrameDecoder和FixedLengthFrameDecoder。这三种是通用方案,不需要自己重新写代码,仅仅需要添加响应的handler到pipeline即可。
当然,我们也可以不使用netty提供的解决方案,而是自己写代码实现,不过这种只针对某些自定义的私有协议,并不是通用解决方案,并不适合所有场景。
接下来的场景如下:
请求格式: 1byte(请求类型) + n bytes(实际内容)
当请求类型确定的情况下,请求的实际内容的长度也是固定的。
客户端和服务端代码和之前的一样,我们只需要在原来的基础上新增一个handler
ch.pipeline().addLast(new OtaTcpDecoder());
OtaTcpDecoder代码如下:
package com.zhuyun.server.handler;
import io.netty.buffer.Unpooled;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.SimpleChannelInboundHandler;
import java.nio.ByteBuffer;
import com.zhuyun.protocols.otatcp.messages.MessageFactory;
import com.zhuyun.protocols.otatcp.messages.OtaFrame;
public class OtaTcpDecoder extends SimpleChannelInboundHandler<byte[]>{
private ChannelHandlerContext currentCtx;
private final MessageFactory messageFactory = new MessageFactory() {
@Override
public void onOtaFrame(OtaFrame frame) {
try {
processFrame(frame);
} catch (Exception e) {
e.printStackTrace();
}
super.onOtaFrame(frame);
}
};
private void processFrame(OtaFrame frame) throws Exception {
ByteBuffer buffer = frame.getFrame();
byte[] array = buffer.array();
currentCtx.fireChannelRead(array);
}
@Override
public void channelRead0(ChannelHandlerContext ctx, byte[] data) throws Exception {// 每次收到消息时被调用
currentCtx = ctx;
messageFactory.getFramer().pushBytes(data);
}
@Override //用来通知handler上一个ChannelRead()是被这批消息中的最后一个消息调用
public void channelReadComplete(ChannelHandlerContext ctx) throws Exception {
//刷新挂起的数据到远端,然后关闭Channel
ctx.writeAndFlush(Unpooled.EMPTY_BUFFER);
}
@Override //在读操作异常被抛出时被调用
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
cause.printStackTrace(); //打印异常堆栈跟踪消息
ctx.close(); //关闭这个Channel
}
@Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
super.channelActive(ctx);
}
@Override
public void channelInactive(ChannelHandlerContext ctx) throws Exception {
super.channelInactive(ctx);
}
}
其中MessageFactory是Framer的工厂类,每当一个连接上来的时候,它就new一个Framer对象。Framer是后面的OtaFrame的构造器,用来新建OtaFrame的对象。MessageFactory注册了一个监听器,每当一个Frame构建完成后,会触发processFrame方法,将一个完整的Frame传递给ServerHandler处理。
下面是MessageFactory的代码:
package com.zhuyun.protocols.otatcp.messages;
import com.zhuyun.protocols.otatcp.Framer;
import com.zhuyun.protocols.otatcp.OtaFramelistener;
public class MessageFactory implements OtaFramelistener {
private Framer framer;
@Override
public void onOtaFrame(OtaFrame frame) {
}
/**
* Constructor with externally created Framer.
* 用外部已创建的Framer创建构造器
* @param framer Framer
*/
public MessageFactory(Framer framer) {
this.setFramer(framer);
framer.registerFrameListener(this);
}
/**
* Default Constructor.
* 预定义构造器
*/
public MessageFactory() {
this.setFramer(new Framer());
framer.registerFrameListener(this);
}
/**
* Framer getter.
*
* @return Framer
*/
public Framer getFramer() {
return framer;
}
/**
* Framer setter.
*
* @param framer Framer
*/
public void setFramer(Framer framer) {
this.framer = framer;
}
}
下面是Framer的代码:
package com.zhuyun.protocols.otatcp;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import com.zhuyun.protocols.otatcp.messages.FiveFrame;
import com.zhuyun.protocols.otatcp.messages.MessageType;
import com.zhuyun.protocols.otatcp.messages.OtaFrame;
import com.zhuyun.protocols.otatcp.messages.ThreeFrame;
public class Framer {
/**
* Mqtt frame listeners list.
*/
private final List<OtaFramelistener> listeners;
/**
* Current processing frame.
*/
private OtaFrame currentFrame;
private int currentFramePayloadLength;
/**
* Default constructor.
*/
public Framer() {
listeners = new ArrayList<>();
}
/**
* Register Mqtt frame listener.
*
* @param listener MqttFramelistener
*/
public void registerFrameListener(OtaFramelistener listener) {
listeners.add(listener);
}
/**
* Process incoming bytes stream.
* Assumes that bytes is unprocessed bytes.
* In case of previous pushBytes() eaten not all bytes on next iterations
* bytes array should starts from unprocessed bytes.
* 处理传入字节流
* 假设这是未经处理的字节
* 假设先前pushBytes()不能处理完所有字节数组,那么下一个字节数组应该开始于未处理的字节
* @param bytes byte[] to push
* @return number of bytes processed from this array.
* @throws Exception
* @throws KaaTcpProtocolException throws in case of protocol errors.
*/
public int pushBytes(byte[] bytes) throws Exception{
// System.out.println("Received bytes: " + Arrays.toString(bytes));
int used = 0;
while (bytes.length > used) {
if (currentFrame == null) {
if ((bytes.length - used) >= 1) { // 1 bytes minimum header length
int intType = bytes[used] & 0xFF;
currentFrame = getFrameByType((byte)intType);
currentFramePayloadLength = getFramePayloadLengthByType((byte)intType);
++used;
} else {
break;
}
}
used += currentFrame.push(bytes, used, currentFramePayloadLength);
if (currentFrame.decodeComplete()) {
callListeners(currentFrame.upgradeFrame());
currentFrame = null;
}
}
return used;
}
/**
* Notify all listeners on new Frame.
*/
private void callListeners(OtaFrame frame) {
for (OtaFramelistener listener : listeners) {
listener.onOtaFrame(frame);
}
}
/**
* Creates specific Kaatcp message by MessageType.
*
* @param type - MessageType of mqttFrame
* @return mqttFrame
* @throws Exception
* @throws KaaTcpProtocolException if specified type is unsupported
*/
private OtaFrame getFrameByType(byte type) throws Exception{
OtaFrame frame = null;
if (type == MessageType.THREE.getType()) {
frame = new ThreeFrame();
}else if (type == MessageType.FIVE.getType()) {
frame = new FiveFrame();
}else {
throw new Exception("Got incorrect messageType format " + type);
}
return frame;
}
private int getFramePayloadLengthByType(byte type) throws Exception{
int length = 0;
if (type == MessageType.THREE.getType()) {
length = ThreeFrame.PAYLOAD_LENGTH;
}else if (type == MessageType.FIVE.getType()) {
length = FiveFrame.PAYLOAD_LENGTH;
}else {
throw new Exception("Got incorrect messageType format " + type);
}
return length;
}
/**
* Reset Framer state by dropping currentFrame.
*/
public void flush() {
currentFrame = null;
}
}
它里面有一个当前Frame的引用currentFrame,还有一个pushBytes的方法在每次接收到数据时会被调用。pushBytes的主要逻辑是:根据第一个字节获取当前的Frame的类型以及长度,然后一个个字节往后处理;若处理完成,则触发前面注册的监听器的方法。
下面是OtaFrame的监听器接口:
package com.zhuyun.protocols.otatcp;
import com.zhuyun.protocols.otatcp.messages.OtaFrame;
public interface OtaFramelistener {
public void onOtaFrame(OtaFrame frame);
}
我们假设目前只有两种类型的Frame请求,ThreeFrame和FiveFrame,都继承了OtaFrame接口。其中,ThreeFrame包含第一个字节(请求类型)总共有3个字节长度,FiveFrame包含第一个字节的总共有5个字节长度。还有一个表示所有Frame类型的枚举类MessageType.
package com.zhuyun.protocols.otatcp.messages;
public enum MessageType {
THREE((byte) 3),
FIVE((byte) 5);
private byte type;
private MessageType(byte type) {
this.type = type;
}
public byte getType() {
return type;
}
}
package com.zhuyun.protocols.otatcp.messages;
public class ThreeFrame extends OtaFrame {
public static final int PAYLOAD_LENGTH = 2;
}
package com.zhuyun.protocols.otatcp.messages;
public class FiveFrame extends OtaFrame {
public static final int PAYLOAD_LENGTH = 4;
}
package com.zhuyun.protocols.otatcp.messages;
import java.nio.ByteBuffer;
public abstract class OtaFrame {
public int PAYLOAD_LENGTH;
protected ByteBuffer buffer;
protected boolean frameDecodeComplete = false;
protected int remainingLength = 0; //完整的帧还剩多少字节需要解析
protected FrameParsingState currentState = FrameParsingState.NONE;
/*
* If adding any filed, don't forget to update MqttFrame(MqttFrame old)
* to clone all fileds.
* 如果你添加任何变量,请不要忘记更新MqttFrame到所有变量中
*/
private MessageType messageType;
protected OtaFrame() {
}
protected OtaFrame(OtaFrame old) {
this.messageType = old.getMessageType();
this.buffer = old.getBuffer();
this.frameDecodeComplete = old.frameDecodeComplete;
this.remainingLength = old.remainingLength;
this.currentState = old.currentState;
}
public MessageType getMessageType() {
return messageType;
}
protected void setMessageType(MessageType messageType) {
this.messageType = messageType;
}
/**
* Return mqtt Frame.
* 返回mqtt帧
* @return ByteBuffer mqtt frame
*/
public ByteBuffer getFrame() {
// if (buffer == null) {
// buffer.position(0);
// }
return buffer;
}
/**
* Return remaining length of mqtt frame, necessary for ByteBuffer size calculation.
* 返回mqtt帧的剩余长度,必须计算ByteBuffer的长度
* @return remaining length of mqtt frame
*/
protected int getRemainingLegth() {
return remainingLength;
}
protected ByteBuffer getBuffer() {
return buffer;
}
private void onFrameDone(){
// System.out.println("Frame (" + getMessageType() + "): payload processed");
if (buffer != null) {
buffer.position(0);
}
frameDecodeComplete = true;
}
private void processByte(int payloadLength) {
if (currentState.equals(FrameParsingState.PROCESSING_PAYLOAD)) {
remainingLength += payloadLength;
if (remainingLength != 0) {
buffer = ByteBuffer.allocate(remainingLength + 1); //包含第一个字节 标志位
} else {
onFrameDone();
}
}
}
/**
* Push bytes of frame.
* 上传帧的字节
* @param bytes the bytes array
* @param position the position in buffer
* @return int used bytes from buffer
*/
public int push(byte[] bytes, int position, int payloadLength){
// position初始位置,pos实际解析的位置
int pos = position;
if (currentState.equals(FrameParsingState.NONE)) {
remainingLength = 0;
currentState = FrameParsingState.PROCESSING_PAYLOAD;
processByte(payloadLength);
buffer.put(bytes, pos-1, 1);
}
while (pos < bytes.length && !frameDecodeComplete) {
if (currentState.equals(FrameParsingState.PROCESSING_PAYLOAD)) {
int bytesToCopy = (remainingLength > bytes.length - pos) ? bytes.length - pos :
remainingLength;
buffer.put(bytes, pos, bytesToCopy);
pos += bytesToCopy;
remainingLength -= bytesToCopy;
if (remainingLength == 0) {
onFrameDone();
}
}
}
return pos - position; //这一次解析了多少个字节
}
/**
* Test if Mqtt frame decode complete.
* 测试mqtt帧解码是否完成
* @return boolean 'true' if decode complete
*/
public boolean decodeComplete() {
return frameDecodeComplete;
}
/**
* Used in case if Frame Class should be changed during frame decode,
* Used for migrate from KaaSync() general frame to specific classes like Sync, Bootstrap.
* Default implementation is to return this.
* 在帧解码期间帧类应该改变情况下使用,
*
* @return new MqttFrame as specific class.
* @throws KaaTcpProtocolException the kaa tcp protocol exception
*/
public OtaFrame upgradeFrame() {
return this;
}
/*
* (non-Javadoc)
* @see java.lang.Object#toString()
*/
protected enum FrameParsingState {
NONE,//没有
PROCESSING_PAYLOAD,//处理有效负荷
}
}
其中,OtaFrame类的主要在于push(byte[] bytes, int position, int payloadLength)这个方法,它是处理前面传入的Frame的主要方法。主要逻辑是:Frame初始状态是FrameParsingState.NONE,这个时候根据Frame的类型给Buffer分配相应长度的空间,然后将第一个字节和这个长度的内容存入Buffer,返回上一层,利用fireChannelRead传递给ServerHandler处理。
再次运行,结果如下:
结果没有粘包了,个数跟请求的数量也是一致的:
至此,基于自定义的私有协议的粘包的解决方案完成。