LengthFieldBasedFrameDecoder解码器自定义协议解决粘包、拆包问题

最新推荐文章于 2024-05-06 16:23:13 发布

易佰菜鸟

最新推荐文章于 2024-05-06 16:23:13 发布

阅读量2.1k

点赞数 1

分类专栏：代码文档文章标签： tcp/ip java netty Decoder 粘包

本文链接：https://blog.csdn.net/han949417140/article/details/122989354

版权

代码文档专栏收录该内容

29 篇文章 2 订阅

订阅专栏

一、介绍

1.1 LengthFieldBasedFrameDecoder作用

LengthFieldBasedFrameDecoder解码器自定义长度解决TCP粘包黏包问题。所以LengthFieldBasedFrameDecoder又称为: 自定义长度解码器

1.2 TCP粘包和黏包现象

TCP粘包是指发送方发送的若干个数据包到接收方时粘成一个包。从接收缓冲区来看，后一个包数据的头紧接着前一个数据的尾。
当TCP连接建立后，Client发送多个报文给Server，TCP协议保证数据可靠性，但无法保证Client发了n个包，服务端也按照n个包接收。Client端发送n个数据包，Server端可能收到n-1或n+1个包。

1.3 为什么出现粘包现象

发送方原因
TCP默认会使用Nagle算法。而Nagle算法主要做两件事：1）只有上一个分组得到确认，才会发送下一个分组；2）收集多个小分组，在一个确认到来时一起发送。所以，正是Nagle算法造成了发送方有可能造成粘包现象。
接收方原因
TCP接收方采用缓存方式读取数据包，一次性读取多个缓存中的数据包。自然出现前一个数据包的尾和后一个收据包的头粘到一起。

1.4 如何解决粘包现象

添加特殊符号，接收方通过这个特殊符号将接收到的数据包拆分开 - DelimiterBasedFrameDecoder特殊分隔符解码器
每次发送固定长度的数据包 - FixedLengthFrameDecoder定长编码器
在消息头中定义长度字段，来标识消息的总长度 - LengthFieldBasedFrameDecoder自定义长度解码器（本文详细介绍此方案）

1.5 LengthFieldBasedFrameDecoder - 6个参数解释

LengthFieldBasedFrameDecoder是自定义长度解码器，所以构造函数中6个参数，基本都围绕那个定义长度域，进行的描述。

maxFrameLength
发送的数据帧最大长度
lengthFieldOffset
定义长度域位于发送的字节数组中的下标。换句话说：发送的字节数组中下标为${lengthFieldOffset}的地方是长度域的开始地方
lengthFieldLength
用于描述定义的长度域的长度。换句话说：发送字节数组bytes时, 字节数组bytes[lengthFieldOffset, lengthFieldOffset+lengthFieldLength]域对应于的定义长度域部分
lengthAdjustment
满足公式: 发送的字节数组发送数据包长度 = 长度域的值 + lengthFieldOffset + lengthFieldLength + lengthAdjustment
initialBytesToStrip
接收到的发送数据包，去除前initialBytesToStrip位
failFast - true
读取到长度域超过maxFrameLength，就抛出一个 TooLongFrameException。false: 只有真正读取完长度域的值表示的字节之后，才会抛出 TooLongFrameException，默认情况下设置为true，建议不要修改，否则可能会造成内存溢出
具体解释可参考：https://blog.csdn.net/liyantianmin/article/details/85603347

二、服务端实现

2.1 协议实体的定义

package com.powernow.usm.netty.protocol;

import io.netty.channel.Channel;
import lombok.Data;

/**
 * 消息载体.
 *
 * 传输模块与服务模块之间双向数据传输载体:
 *
 *                   MessageHolder
 * Service Module <----------------> Transport Module
 *
 */
@Data
public class MessageHolder {

    // 消息标志  0x01:Client --> Server
    /**
     * 消息标志    0x01:请求 Client --> Server
     *           0x02:响应 Server --> Client
     *           0x03:通知 Server --> Client  e.g.消息转发
     */
    private byte sign;
    // 消息类型
    /**
     * 消息类型 0x15: 个人消息
     *        0x16： 群组消息
     */
    private byte type;
    // 响应状态
    private byte status;
    // Json消息体
    private String body;
    // 接收到消息的通道
    private Channel channel;
}

2.2 自定义LengthFieldBasedFrameDecoder解码器

package com.powernow.usm.netty.handler;

import com.powernow.usm.netty.protocol.MessageHolder;
import com.powernow.usm.netty.protocol.ProtocolHeader;
import io.netty.buffer.ByteBuf;
import io.netty.channel.ChannelHandlerContext;
import io.netty.handler.codec.ByteToMessageDecoder;
import io.netty.handler.codec.LengthFieldBasedFrameDecoder;
import lombok.extern.slf4j.Slf4j;

import java.util.List;

/**
 * LengthFieldBasedFrameDecoder 解决粘包问题，https://www.jianshu.com/p/c90ec659397c
 * 解码Handler.
 *
 *                                       Jelly Protocol
 *  __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __
 * |           |           |           |           |              |                          |
 *       2           1           1           1            4               Uncertainty
 * |__ __ __ __|__ __ __ __|__ __ __ __|__ __ __ __|__ __ __ __ __|__ __ __ __ __ __ __ __ __|
 * |           |           |           |           |              |                          |
 *     Magic        Sign        Type       Status     Body Length         Body Content
 * |__ __ __ __|__ __ __ __|__ __ __ __|__ __ __ __|__ __ __ __ __|__ __ __ __ __ __ __ __ __|
 *
 * 协议头9个字节定长
 *     Magic      // 数据包的验证位，short类型
 *     Sign       // 消息标志，请求／响应／通知，byte类型
 *     Type       // 消息类型，登录／发送消息等，byte类型
 *     Status     // 响应状态，成功／失败，byte类型
 *     BodyLength // 协议体长度，int类型
 */
@Slf4j
public class ProtocolDecoder extends LengthFieldBasedFrameDecoder {
    
    public ProtocolDecoder(int maxFrameLength, int lengthFieldOffset, int lengthFieldLength, int lengthAdjustment, int initialBytesToStrip, boolean failFast) {
        super(maxFrameLength, lengthFieldOffset, lengthFieldLength,lengthAdjustment,initialBytesToStrip,failFast);
    }

    @Override
    protected Object decode(ChannelHandlerContext ctx, ByteBuf in) throws Exception {
        //在这里调用父类的方法,实现指得到想要的部分,我在这里全部都要,也可以只要body部分
        in = (ByteBuf) super.decode(ctx,in);

        if(in == null){
            return null;
        }
        if(in.readableBytes() < ProtocolHeader.HEADER_LENGTH){
            throw new Exception("数据包长度小于协议头长度");
        }
        short magic = in.readShort();
        // 开始解码
        byte sign = in.readByte();
        byte type = in.readByte();
        byte status = in.readByte();
        // 确认消息体长度
        int length = in.readInt();
        
        if(in.readableBytes()!=length){
            throw new Exception("消息体不一致");
        }
        //读取body
        byte []bytes = new byte[in.readableBytes()];
        in.readBytes(bytes);
        MessageHolder messageHolder = new MessageHolder();
        messageHolder.setSign(sign);
        messageHolder.setType(type);
        messageHolder.setStatus(status);
        messageHolder.setBody(new String(bytes, "utf-8"));
        return messageHolder;

    }
}

2.3 服务端Hanlder

package com.powernow.usm.netty.handler;

import com.powernow.usm.netty.protocol.MessageHolder;
import com.powernow.usm.netty.protocol.ProtocolHeader;
import com.powernow.usm.netty.queue.TaskQueue;
import io.netty.channel.Channel;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.ChannelInboundHandlerAdapter;
import lombok.extern.slf4j.Slf4j;

import java.util.concurrent.BlockingQueue;

/**
 * 最终接收数据的Handler，将待处理数据放入阻塞队列中，由服务模块take and deal.
 *
 */
@Slf4j
public class AcceptorHandler extends ChannelInboundHandlerAdapter {

    private final BlockingQueue<MessageHolder> taskQueue;

    public AcceptorHandler() {
        taskQueue = TaskQueue.getQueue();
    }

    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
        if (msg instanceof MessageHolder) {
            MessageHolder messageHolder = (MessageHolder) msg;
            // 指定Channel
            messageHolder.setChannel(ctx.channel());
            // 添加到任务队列
            boolean offer = taskQueue.offer(messageHolder);
            log.info("TaskQueue添加任务: taskQueue={},message = {}" , taskQueue.size(),messageHolder.toString());
            if (!offer) {
                // 服务器繁忙
                log.warn("服务器繁忙，拒绝服务");
                // 繁忙响应
                response(ctx.channel(), messageHolder.getSign());
            }
        } else {
            throw new IllegalArgumentException("msg is not instance of MessageHolder");
        }
    }


    @Override
    public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
        cause.printStackTrace();
        ctx.close();
    }

    /**
     * 服务器繁忙响应
     *
     * @param channel
     * @param sign
     */
    private void response(Channel channel, byte sign) {
        MessageHolder messageHolder = new MessageHolder();
        messageHolder.setSign(ProtocolHeader.RESPONSE);
        messageHolder.setType(sign);
        messageHolder.setStatus(ProtocolHeader.SERVER_BUSY);
        messageHolder.setBody("");
        channel.writeAndFlush(messageHolder);
    }
}

2.4 服务端实现

package com.powernow.usm.netty;

import com.powernow.usm.netty.handler.AcceptorHandler;
import com.powernow.usm.netty.handler.ProtocolDecoder;
import com.powernow.usm.netty.handler.ProtocolEncoder;
import io.netty.bootstrap.ServerBootstrap;
import io.netty.channel.ChannelFuture;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.ChannelOption;
import io.netty.channel.EventLoopGroup;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioServerSocketChannel;
import io.netty.handler.timeout.IdleStateHandler;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;

import javax.annotation.PostConstruct;

/**
 * @destription
 */
@Slf4j
@Component
public class NettyServer {

    @Value("${netty.port}")
    private Integer port;

    private static final int MAX_FRAME_LENGTH = 1024 * 1024;  //最大长度
    private static final int LENGTH_FIELD_LENGTH = 4;  //长度字段所占的字节数
    private static final int LENGTH_FIELD_OFFSET = 5;  //长度偏移
    private static final int LENGTH_ADJUSTMENT = 0;
    private static final int INITIAL_BYTES_TO_STRIP = 0;

    @PostConstruct
    public void start() throws Exception {
        EventLoopGroup bossGroup = new NioEventLoopGroup();

        EventLoopGroup group = new NioEventLoopGroup();
        try {
            ServerBootstrap sb = new ServerBootstrap();
            sb.group(group, bossGroup) // 绑定线程池
                    .channel(NioServerSocketChannel.class) // 指定使用的channel
                    .localAddress(port)// 绑定监听端口
                    .option(ChannelOption.SO_BACKLOG, 1024)
                    .childOption(ChannelOption.SO_KEEPALIVE, true)
                    .childHandler(new ChannelInitializer<SocketChannel>() { // 绑定客户端连接时候触发操作
                        @Override
                        protected void initChannel(SocketChannel ch) throws Exception {
                            log.info("收到新的客户端连接: {}",ch.toString());
                            ch.pipeline().addLast("ProtocolDecoder", new ProtocolDecoder(MAX_FRAME_LENGTH,LENGTH_FIELD_OFFSET,LENGTH_FIELD_LENGTH,LENGTH_ADJUSTMENT,INITIAL_BYTES_TO_STRIP,true));
//                            ch.pipeline().addLast("ProtocolDecoder", new ProtocolDecoder1(1024 * 8,5,4));
                            ch.pipeline().addLast("ProtocolEncoder", new ProtocolEncoder());
                            ch.pipeline().addLast("IdleStateHandler", new IdleStateHandler(6, 0, 0));
                            ch.pipeline().addLast("AcceptorHandler", new AcceptorHandler());
                        }
                    });
            ChannelFuture cf = sb.bind().sync(); // 服务器异步创建绑定
            System.out.println(NettyServer.class + " 启动正在监听： " + cf.channel().localAddress());
            cf.channel().closeFuture().sync(); // 关闭服务器通道
        } catch (InterruptedException e) {
            log.warn("Netty绑定异常", e);
        } finally {
            group.shutdownGracefully().sync(); // 释放线程池资源
            bossGroup.shutdownGracefully().sync();
        }
    }
}

自定义解码器参数解释

new ProtocolDecoder(MAX_FRAME_LENGTH,LENGTH_FIELD_OFFSET,LENGTH_FIELD_LENGTH,LENGTH_ADJUSTMENT,INITIAL_BYTES_TO_STRIP,true)
maxFrameLength: 帧的最大长度
lengthFieldOffset length: 字段偏移的地址
lengthFieldLength length;字段所占的字节长
lengthAdjustment: 修改帧数据长度字段中定义的值，可以为负数 因为有时候我们习惯把头部记入长度,若为负数,则说明要推后多少个字段
initialBytesToStrip: 解析时候跳过多少个长度
failFast; 为true，当frame长度超过maxFrameLength时立即报TooLongFrameException异常，为false，读取完整个帧再报异

三、客户端实现

3.1 自定义客户端编码器

package com.powernow.usm.netty.handler;

import cn.hutool.json.JSONUtil;
import com.powernow.usm.config.BizException;
import com.powernow.usm.netty.protocol.MessageHolder;
import com.powernow.usm.netty.protocol.ProtocolHeader;
import io.netty.buffer.ByteBuf;
import io.netty.channel.ChannelHandlerContext;
import io.netty.handler.codec.MessageToByteEncoder;
import lombok.Data;
import lombok.extern.slf4j.Slf4j;


/**
 * 编码Handler.
 *
 *                                       Jelly Protocol
 *  __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __
 * |           |           |           |           |              |                          |
 *       2           1           1           1            4               Uncertainty
 * |__ __ __ __|__ __ __ __|__ __ __ __|__ __ __ __|__ __ __ __ __|__ __ __ __ __ __ __ __ __|
 * |           |           |           |           |              |                          |
 *     Magic        Sign        Type       Status     Body Length         Body Content
 * |__ __ __ __|__ __ __ __|__ __ __ __|__ __ __ __|__ __ __ __ __|__ __ __ __ __ __ __ __ __|
 *
 * 协议头9个字节定长
 *     Magic      // 数据包的验证位，short类型
 *     Sign       // 消息标志，请求／响应／通知，byte类型
 *     Type       // 消息类型，登录／发送消息等，byte类型
 *     Status     // 响应状态，成功／失败，byte类型
 *     BodyLength // 协议体长度，int类型
 *
 *
 */
@Slf4j
public class ProtocolEncoder extends MessageToByteEncoder<MessageHolder> {

    @Override
    protected void encode(ChannelHandlerContext ctx, MessageHolder msg, ByteBuf out) throws Exception {
        String body = msg.getBody();
        if (msg == null || msg.getBody() == null) {
            throw new BizException("msg == null");
        }
        // 编码
        byte[] bytes = body.getBytes("utf-8");

        out.writeShort(ProtocolHeader.MAGIC)
                .writeByte(msg.getSign())
                .writeByte(msg.getType())
                .writeByte(msg.getStatus())
                .writeInt(bytes.length)
                .writeBytes(bytes);

    }
}

3.2 客户端Handler

package com.powernow.usm.netty.client;


import com.powernow.usm.dto.Message;
import com.powernow.usm.netty.protocol.MessageHolder;
import com.powernow.usm.netty.protocol.ProtocolHeader;
import com.powernow.usm.utils.Serializer;
import io.netty.channel.Channel;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.ChannelInboundHandlerAdapter;
import io.netty.channel.ChannelPromise;
import lombok.extern.slf4j.Slf4j;

import java.util.concurrent.TimeUnit;

/**
 * 最终接收数据的Handler.
 */
@Slf4j
public class AcceptorHandler extends ChannelInboundHandlerAdapter {

    private ChannelHandlerContext ctx;

    public AcceptorHandler(){}

    @Override
    public void channelActive(ChannelHandlerContext ctx) throws Exception {
        System.out.println("channel 活跃");
        this.ctx = ctx;
		// 客户端连续发送10条消息
        for (int i = 0; i < 10; i++) {
            Message message = new Message();
            message.setSender("sender" + i);
            message.setReceiver( "receiver" + i);
            message.setContent("hello receiver " + i + ", i am sender" + i);
            message.setTime(System.currentTimeMillis());
            MessageHolder messageHolder = new MessageHolder();
            messageHolder.setSign(ProtocolHeader.REQUEST);
            messageHolder.setType(ProtocolHeader.PERSON_MESSAGE);
            messageHolder.setStatus((byte) 0);
            messageHolder.setBody(Serializer.serialize(message));
            Channel channel = ctx.channel();
            channel.writeAndFlush(messageHolder);
        }
    }

    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
        System.out.println("读取完毕 活跃");
        if (msg instanceof MessageHolder) {
            MessageHolder messageHolder = (MessageHolder) msg;
            log.info(messageHolder.toString());
            // 处理消息
//            Dispatcher.dispatch(messageHolder);
        } else {
            throw new IllegalArgumentException("msg is not instance of MessageHolder");
        }
    }
}

3.3 客户端实现

package com.powernow.usm.netty.client;

import com.powernow.usm.netty.handler.ProtocolDecoder;
import com.powernow.usm.netty.handler.ProtocolEncoder;
import io.netty.bootstrap.Bootstrap;
import io.netty.channel.ChannelFuture;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.EventLoopGroup;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioSocketChannel;
import io.netty.handler.timeout.IdleStateHandler;

import java.net.InetSocketAddress;

/**
 * @destription
 */
public class NettyClient {
    private final String host;
    private final int port;


    private static final int MAX_FRAME_LENGTH = 1024 * 1024;  //最大长度
    private static final int LENGTH_FIELD_LENGTH = 4;  //长度字段所占的字节数
    private static final int LENGTH_FIELD_OFFSET = 5;  //长度偏移
    private static final int LENGTH_ADJUSTMENT = 0;
    private static final int INITIAL_BYTES_TO_STRIP = 0;

    public NettyClient() {
        this(12345);
    }

    public NettyClient(int port) {
        this("localhost", port);
    }

    public NettyClient(String host, int port) {
        this.host = host;
        this.port = port;
    }

    public void start() throws Exception {
        EventLoopGroup group = new NioEventLoopGroup();
        try {
            Bootstrap b = new Bootstrap();
            b.group(group) // 注册线程池
                    .channel(NioSocketChannel.class) // 使用NioSocketChannel来作为连接用的channel类
                    .remoteAddress(new InetSocketAddress(this.host, this.port)) // 绑定连接端口和host信息
                    .handler(new ChannelInitializer<SocketChannel>() { // 绑定连接初始化器
                        @Override
                        protected void initChannel(SocketChannel ch) throws Exception {
                            System.out.println("连接connected...");
                            ch.pipeline().addLast("ProtocolDecoder", new ProtocolDecoder(MAX_FRAME_LENGTH,LENGTH_FIELD_OFFSET,LENGTH_FIELD_LENGTH,LENGTH_ADJUSTMENT,INITIAL_BYTES_TO_STRIP,true));//防止粘包处理
                            ch.pipeline().addLast("ProtocolEncoder", new ProtocolEncoder());
                            ch.pipeline().addLast("IdleStateHandler", new IdleStateHandler(0, 5, 0));
                            ch.pipeline().addLast("ReaderHandler", new AcceptorHandler());
                        }
                    });
            System.out.println("created..");

            ChannelFuture cf = b.connect().sync(); // 异步连接服务器
            System.out.println("connected..."); // 连接完成

            cf.channel().closeFuture().sync(); // 异步等待关闭连接channel
            System.out.println("closed.."); // 关闭完成
        } finally {
            group.shutdownGracefully().sync(); // 释放线程池资源
        }
    }

    public static void main(String[] args) throws Exception {
        new NettyClient("127.0.0.1", 12345).start();// 连接127.0.0.1/12345，并启动
        System.out.println("===================================");
    }
}