Netty权威指南阅读笔记(编解码)

最新推荐文章于 2021-08-18 20:52:20 发布

qq_42913304

最新推荐文章于 2021-08-18 20:52:20 发布

阅读量175

点赞数

分类专栏： Netty

本文链接：https://blog.csdn.net/qq_42913304/article/details/109397727

版权

Netty 专栏收录该内容

1 篇文章 0 订阅

订阅专栏

6.1 Java序列化的缺点

6.1.1 无法跨语言

无法跨语言是java序列化最致命的问题对于跨进程的服务调用服务提供者可能会使用其它语言开发当我们需要和异构语言进行交互时 java序列化就难以胜任对于java序列化技术是java语言内部的私有协议别的语言无法进行反序列化严重阻碍它的应用目前几乎所有流行的Java rpc框架都没有使用java序列化作为编解码框架原因就是因为它无法跨语言

6.1.2 序列化后的码流太大

public class UserInfo implements Serializable {
    private String username;
    private int userId;

    public UserInfo buildUserId(Integer userId){
        this.userId = userId;
        return this;
    }
    public UserInfo buildUserName(String username){
        this.username = username;
        return this;
    }
    public final String getUsername(){
        return username;
    }

    public final void setUsername(String username){
        this.username = username;
    }
    public final  int getUserId(){
        return userId;
    }

    public final  void setUserId(int userId){
        this.userId = userId;
    }

    public byte[] codeC(){
        ByteBuffer buffer = ByteBuffer.allocate(1024);
        byte[] value = this.username.getBytes();
        buffer.putInt(value.length);
        buffer.put(value);
        buffer.putInt(this.userId);
        buffer.flip();
        value = null;
        byte[] result = new byte[buffer.remaining()];
        buffer.get(result);
        return result;
    }

    public static class TestUserInfo{
        public static void main(String[] args) throws IOException {
            UserInfo info = new UserInfo();
            info.buildUserId(100).buildUserName("welcome netty");
            ByteArrayOutputStream bos = new ByteArrayOutputStream();
            ObjectOutputStream os = new ObjectOutputStream(bos);
            os.writeObject(info);
            os.flush();
            os.close();
            byte[] b = bos.toByteArray();
            System.out.println("the jdk serializable length is " + b.length);
            bos.close();
            System.out.println("----------------");
            System.out.println("the byte array length is " + info.codeC().length);
        }
    }
}

在这里插入图片描述
采用jdk序列化编号后的二进制数组大小竟然是二进制编码的5.29倍在同等情况编码后的字节数越大储存越占空间存储的硬件成本就越高

6.1.3 序列化性能太低

新增方法

public byte[] codeD(ByteBuffer buffer){
        buffer.clear();
        byte[] value = this.username.getBytes();
        buffer.putInt(value.length);
        buffer.put(value);
        buffer.putInt(this.userId);
        buffer.flip();
        value = null;
        byte[] result = new byte[buffer.remaining()];
        buffer.get(result);
        return result;
    }

 public static class TestUserInfo{
        public static void main(String[] args) throws IOException {
            UserInfo info = new UserInfo();
            info.buildUserId(100).buildUserName("welcome netty");
            int loop = 1000000;
            ByteArrayOutputStream bos = null;
            ObjectOutputStream os = null;
            long startTime = System.currentTimeMillis();
            for (int i = 0; i < loop; i ++){
                bos = new ByteArrayOutputStream();
                os = new ObjectOutputStream(bos);
                os.writeObject(info);
                os.flush();
                os.close();
                byte[] b = bos.toByteArray();
                bos.close();;
            }
            long endTime = System.currentTimeMillis();
            System.out.println("the jdk time ：" + (endTime - startTime) + "ms");
            ByteBuffer buffer = ByteBuffer.allocate(1024);
            startTime = System.currentTimeMillis();
            for (int i = 0; i <loop; i ++){
                byte[] b = info.codeD(buffer);
            }
            endTime = System.currentTimeMillis();
            System.out.println("the byte array time: " + (endTime - startTime) + "ms");
        }
    }

在这里插入图片描述
java原生序列化和二进制编码的对比可见原生序列化性能实在太差

7.1 Netty Java序列化

POJO

public class SubscribeResp implements Serializable {

 
    private static final long serialVersionUID = 1L;

    private int subReqID;

    private int respCode;

    private String desc;
}

public class SubscribeReq implements Serializable {

    /**
     * 默认的序列号ID
     */
    private static final long serialVersionUID = 1L;

    private int subReqID;

    private String userName;

    private String productName;

    private String phoneNumber;

    private String address;
}

public class SubscribeServer {
    public void bind(int port) throws Exception {
        // 配置服务端的NIO线程组
        EventLoopGroup bossGroup = new NioEventLoopGroup();
        EventLoopGroup workerGroup = new NioEventLoopGroup();
        try {
            ServerBootstrap b = new ServerBootstrap();
            b.group(bossGroup, workerGroup)
                    .channel(NioServerSocketChannel.class)
                    .option(ChannelOption.SO_BACKLOG, 100)
                    .handler(new LoggingHandler(LogLevel.INFO))
                    .childHandler(new ChannelInitializer<SocketChannel>() {
                        @Override
                        public void initChannel(SocketChannel ch) {
                            ch.pipeline()
                                    .addLast(
                                            new ObjectDecoder(
                                                    1024 * 1024,
                                                    ClassResolvers
                                                            .weakCachingConcurrentResolver(this
                                                                    .getClass()
                                                                    .getClassLoader())));
                            ch.pipeline().addLast(new ObjectEncoder());
                            ch.pipeline().addLast(new SubscribeHandler());
                        }
                    });

            // 绑定端口，同步等待成功
            ChannelFuture f = b.bind(port).sync();

            // 等待服务端监听端口关闭
            f.channel().closeFuture().sync();
        } finally {
            // 优雅退出，释放线程池资源
            bossGroup.shutdownGracefully();
            workerGroup.shutdownGracefully();
        }
    }


    public static class SubscribeHandler extends ChannelHandlerAdapter {
        @Override
        public void channelRead(ChannelHandlerContext ctx, Object msg)
                throws Exception {
            SubscribeReq req = (SubscribeReq) msg;
            System.out.println("message : " + req);
            ctx.writeAndFlush(req);
        }

        private SubscribeResp resp(int subReqID) {
            SubscribeResp resp = new SubscribeResp();
            resp.setSubReqID(subReqID);
            resp.setRespCode(0);
            resp.setDesc("Success!");
            return resp;
        }

        @Override
        public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
            cause.printStackTrace();
            ctx.close();// 发生异常，关闭链路
        }
    }

    public static void main(String[] args) throws Exception {
        new SubscribeServer().bind(8888);
    }
}

客户端

public class SubscribeClient {
    public void connect(int port, String host) throws Exception {
        // 配置客户端NIO线程组
        EventLoopGroup group = new NioEventLoopGroup();
        try {
            Bootstrap b = new Bootstrap();
            b.group(group).channel(NioSocketChannel.class)
                    .option(ChannelOption.TCP_NODELAY, true)
                    .handler(new ChannelInitializer<SocketChannel>() {
                        @Override
                        public void initChannel(SocketChannel ch)
                                throws Exception {
                            ch.pipeline().addLast(
                                    new ObjectDecoder(1024, ClassResolvers
                                            .cacheDisabled(this.getClass()
                                                    .getClassLoader())));
                            ch.pipeline().addLast(new ObjectEncoder());
                            ch.pipeline().addLast(new SubscribeClientHandler());
                        }
                    });

            // 发起异步连接操作
            ChannelFuture f = b.connect(host, port).sync();

            // 当代客户端链路关闭
            f.channel().closeFuture().sync();
        } finally {
            // 优雅退出，释放NIO线程组
            group.shutdownGracefully();
        }
    }


    public static void main(String[] args) throws Exception {
        new SubscribeClient().connect(8888,"127.0.0.1");
    }
}

public class SubscribeClientHandler extends ChannelHandlerAdapter {

    @Override
    public void channelActive(ChannelHandlerContext ctx) {
        for (int i = 0; i < 10; i++) {
            ctx.write(subReq(i));
        }
        ctx.flush();
    }

    private SubscribeReq subReq(int i) {
        SubscribeReq req = new SubscribeReq();
        req.setAddress("中国");
        req.setPhoneNumber("110");
        req.setProductName("Hello word");
        req.setSubReqID(i);
        req.setUserName("Li");
        return req;
    }

    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg)
            throws Exception {
        System.out.println("Receive server response : [" + msg + "]");
    }

    @Override
    public void channelReadComplete(ChannelHandlerContext ctx) throws Exception {
        ctx.flush();
    }

    @Override
    public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
        cause.printStackTrace();
        ctx.close();
    }
}

Netty 提供的 ObjectEncoder 编码器和 ObjectDecoder 解码器实现对普通 POJO 对象的序列化通过使用 Netty 的 Java 序列化编解码 handler, 用户通过短短的几行代码，就能完成POJO 的序列化和反序列化在业务处理 handler 中，用户只需要将精力聚焦在业务逻辑的实现上不需要关心底层的编解码细节这极大地提升了幵发效率

8.1 Google Protobuf 编解码

Google 的 Protobuf 在业界非常流行 Protobuf 的优点
跨语言支持多种语言包括 C++ Java 和 Python
编码后的消息更小更加有利于存储和传输
编解码的性能非常高

8.1.1 Protobuf使用

下载地址

编写.proto文件

syntax="proto2";
option java_outer_classname = "SubscribeRespProto";//生成的数据访问类的类名  
message SubscribeResp {  
  required int32 subRespId = 1; 
  required int32 respCode = 2;
  required string desc = 3;
}

syntax="proto2";
option java_outer_classname = "SubscribeReqProto";//生成的数据访问类的类名
message SubscribeReq {  
  required int32 subReqId = 1; 
  required string username = 2;
  required string productName = 3;
  required string address = 4;
}

生成java文件
-I=源地址 --java_out=目标地址 xxx.proto

protoc.exe -I=F:\code --java_out=F:\code\code F:\code\Sub1.proto

导入jar

<dependency>
    <groupId>com.google.protobuf</groupId>
    <artifactId>protobuf-java</artifactId>
    <version>3.13.0</version>
</dependency>

8.1.2 Netty Protobuf 服务端开发

更新代码

  @Override
                        public void initChannel(SocketChannel ch) {
                            //添加 ProtobufVarint32FrameDecoder用于半包处理
                          ch.pipeline()
                                  .addLast(new ProtobufVarint32FrameDecoder());
                          //添加 ProtobufDecoder 解码器 它的参数是 com.google.protobuf.MessageLite 就是要告诉 ProtobufDecoder 需要解码的目标类是什么
                          ch.pipeline().addLast(new ProtobufDecoder(SubscribeReqProto.SubscribeReq.getDefaultInstance()));
                          ch.pipeline().addLast(new ProtobufVarint32LengthFieldPrepender());
                          ch.pipeline().addLast(new ProtobufEncoder());
                          ch.pipeline().addLast(new SubscribeHandler());
                        }


  @Override
        public void channelRead(ChannelHandlerContext ctx, Object msg)
                throws Exception {
            // 由于 ProtobufDecoder 已经对消息进行了自动解码， 因此接收到的订购请求消息可以直接使用
            SubscribeReqProto.SubscribeReq req = (SubscribeReqProto.SubscribeReq) msg;
            System.out.println("message : " + req);
            //由于使用了ProtobufEncode所以不需要对 SubscribeRespProto .SubscribeResp 进行手工编码
            ctx.writeAndFlush(this.resp(9999));
        }

        private SubscribeRespProto.SubscribeResp resp(int subReqID) {
            SubscribeRespProto.SubscribeResp.Builder builder =
                    SubscribeRespProto.SubscribeResp.newBuilder();
            builder.setSubRespId(subReqID);
            builder.setRespCode(0);
            builder.setDesc("success!");
            return builder.build();
        }

客户端

   @Override
                        public void initChannel(SocketChannel ch)
                                throws Exception {
                            ch.pipeline().addLast(new ProtobufVarint32FrameDecoder());
                            ch.pipeline().addLast(new ProtobufDecoder(SubscribeRespProto.SubscribeResp.getDefaultInstance()));
                            ch.pipeline().addLast(new ProtobufVarint32LengthFieldPrepender());
                            // 编码
                            ch.pipeline().addLast(new ProtobufEncoder());
                            ch.pipeline().addLast(new SubscribeClientHandler());
                        }



   private SubscribeReqProto.SubscribeReq subReq(int i) {
        SubscribeReqProto.SubscribeReq.Builder
                req = SubscribeReqProto.SubscribeReq.newBuilder();
        req.setAddress("中国");
        req.setProductName("Hello word");
        req.setSubReqId(i);
        req.setUsername("Jack");
        return req.build();
    }

8.1.3 Protobuf 的使用注意事项

ProtobufDecoder 仅仅负责解码，它不支持读半包。因此，在 ProtobufDecoder 前面，一定要有能够处理读半包的解码器有三种方式可以选择
使用 Netty 提供的ProtobufVarint32FrameDecodet它可以处理半包信息
继承 Netty 提供的通用半包解码器 LengthFieldBasedFrameDecoder
继承 ByteToMessageDecoder 类自己处理半包消息
如果你只使用 ProtobufDecoder 解码器而忽略对半包消息的处理程序是不能正常工作的