Netty学习笔记

NIO基础

NIO：non-blocing io 非阻塞io

NIO的三大组件：channel、buffer、selector

ByteBuffer的3个重要属性：position（读写指针）、limit（限制指针）、capacity（容量）

读写过程图：

buffer的写入方式

buffer的读取方式

buffer和字符串的转换

//1.字符串转ByteBuffer，使用后还是写状态
ByteBuffer buffer = ByteBuffer.allocate(16);
buffer.put("FED126".getBytes());

//2.Charset，使用后会变成读状态
ByteBuffer buffer2 = StandardCharsets.UTF_8.encode("qwe");

//3.wrap，使用后会变成读状态
ByteBuffer buffer3 = ByteBuffer.wrap("zxc".getBytes());

//buffer转字符串
String s = StandardCharsets.UTF_8.decode(buffer2).toString();

文件编程

FileChannel

/**
* 通道数据传输
*/
public static void testTranform(){
    try (
        FileChannel from = new FileInputStream("fed.txt").getChannel();
        FileChannel to = new FileOutputStream("to.txt").getChannel();
    ) {
        long size = from.size();
        for (long left = size; left > 0;){
            left -= from.transferTo((size - left), left, to);
        }
    } catch (IOException e) {
        e.printStackTrace();
    }
}

Path

Files

遍历文件夹和文件

/**
* 遍历文件夹和文件
*/
public static void testWalkFileTree() throws IOException {
    AtomicInteger dirCount = new AtomicInteger();
    AtomicInteger fileCount = new AtomicInteger();
    Files.walkFileTree(Paths.get("D:\\maven-repository"), new SimpleFileVisitor<Path>(){
        @Override
        public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs) throws IOException {
            System.out.println(dir);
            dirCount.incrementAndGet();
            return super.preVisitDirectory(dir, attrs);
        }

        @Override
        public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IOException {
            System.out.println(file);
            fileCount.incrementAndGet();
            return super.visitFile(file, attrs);
        }
    });
    System.out.println(dirCount + "："+ fileCount);
}

事件类型：accept-有连接时触发、connect-客户端连接后触发、read-可读事件、write-可写事件

阻塞和非阻塞服务IO的使用：

public class Service {
    public static void main(String[] args) throws IOException {
        //0.创建ByteBuffer存储数据
        ByteBuffer buffer = ByteBuffer.allocate(16);
        //1.创建服务器
        ServerSocketChannel ssc = ServerSocketChannel.open();
        ssc.configureBlocking(false); //非阻塞
        //2.绑定监听端口
        ssc.bind(new InetSocketAddress(8086));
        //3.连接集合
        List<SocketChannel> channels = new ArrayList<>();
        while (true){
            System.out.println("服务启动....");
            //4.建立和客户端的连接
            SocketChannel sc = ssc.accept();
            sc.configureBlocking(false);
            System.out.println("客户端连接....");
            channels.add(sc);
            for (SocketChannel channel : channels){
                //5.接受客户端发送的数据
                channel.read(buffer); //阻塞方法，线程停止运行
                buffer.flip();
                while(buffer.hasRemaining()){ //是否还有剩余未读的数据
                    System.out.println((char)buffer.get());
                }
                buffer.clear();
            }
        }
    }
}

selector的使用：

public class TestSelector {
    public static void main(String[] args) throws IOException {
        //创建selector
        Selector selector = Selector.open();

        //1.创建服务器
        ServerSocketChannel ssc = ServerSocketChannel.open();
        ssc.configureBlocking(false); //非阻塞

        //建立selector和channel的联系，通过事件驱动的方式
        SelectionKey sscKey = ssc.register(selector, 0, null);
        //设置关注的事件
        sscKey.interestOps(SelectionKey.OP_ACCEPT);

        //2.绑定监听端口
        ssc.bind(new InetSocketAddress(8086));
        while (true){
            System.out.println("线程启动...");
            //没有事件，线程阻塞
            selector.select();
            //处理事件，要么取消，要么处理
            Iterator<SelectionKey> iterator = selector.selectedKeys().iterator();
            while (iterator.hasNext()){
                //只是复制出来的key
                SelectionKey key = iterator.next();
                //区分事件类型
                if(key.isAcceptable()){
                    ServerSocketChannel channel = (ServerSocketChannel)key.channel();
                    SocketChannel sc = channel.accept(); //没有事件处理会返回null
                    sc.configureBlocking(false);
                    SelectionKey scKey = sc.register(selector, 0, null);
                    scKey.interestOps(SelectionKey.OP_READ);
                }else if(key.isReadable()){
                    try {
                        SocketChannel channel = (SocketChannel)key.channel();
                        ByteBuffer buffer = ByteBuffer.allocate(16);
                        int read = channel.read(buffer);//阻塞方法，线程停止运行
                        if(read == -1){ //正常断开
                            System.out.println("线程正常断开...");
                            key.cancel();
                            continue;
                        }
                        buffer.flip();
                        System.out.println(Charset.defaultCharset().decode(buffer).toString());
                        buffer.clear();
                    }catch (Exception e){
                        System.out.println("线程异常断开...");
                        key.cancel(); //客户端断开需要将key在真正的keys集合中移除
                    }
                }
                //用完要删除key
                iterator.remove();
            }
        }
    }
}

ByteBuffer的大小分配

监听channel事件

获取cpu的个数

NIO和BIO的区别

IO模型

注意：异步是没有阻塞情况的，异步是非阻塞的

Netty：异步的事件驱动的网络应用框架，用于网络编程，底层使用nio的方式开发

netty简单服务端和客户端通信代码

public class HelloService {
    public static void main(String[] args) {
        //1.启动器，负责组装netty组件，启动服务器
        new ServerBootstrap()
                //2.BossEvenLoop，WorkerEvenLoop(selector,thread)
                .group(new NioEventLoopGroup())
                //3.选择服务器的ServerSocketChannel实现
                .channel(NioServerSocketChannel.class)
                //4.boss 负责处理连接 worker(child)负责处理读写，决定worker能执行哪些操作
                .childHandler(
                    //5.channel代表和客户端进行数据读写的通道，Initializer初始化，负责添加别的handler
                    new ChannelInitializer<NioSocketChannel>() {
                    //连接建立后调用initChannel方法
                    @Override
                    protected void initChannel(NioSocketChannel ch) throws Exception {
                        //6.添加具体的handler
                        ch.pipeline().addLast(new StringDecoder()); //讲byte转换为字符串
                        ch.pipeline().addLast(new ChannelInboundHandlerAdapter(){ //自定义handler
                            @Override //读事件
                            public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
                                System.out.println(msg);
                            }
                        });
                    }
                })
                .bind(8088); //服务器的端口
    }
}

public class HelloClient {
    public static void main(String[] args) throws InterruptedException {
        //启动类
        new Bootstrap()
                //添加EvenLoop
                .group(new NioEventLoopGroup())
                //选择客户端channel实现
                .channel(NioSocketChannel.class)
                //添加处理器
                .handler(new ChannelInitializer<NioSocketChannel>() {
                    //连接建立后调用initChannel方法
                    @Override
                    protected void initChannel(NioSocketChannel ch) throws Exception {
                        ch.pipeline().addLast(new StringEncoder());
                    }
                })
                //连接服务器
                .connect(new InetSocketAddress("localhost", 8088))
                .sync()
                .channel()
                .writeAndFlush("fed1231564");
    }
}

netty中的组件：EventLoop、Channel、Future&Promise、Handler&Piepeline、ByteBuf

EventLoop：

EventLoop的简单使用：

public class TestEventLoop {
    public static void main(String[] args) {
        //1.创建事件循环组
        EventLoopGroup group = new NioEventLoopGroup(2); // io 事件，普通任务，定时任务
        //2.获取下一事件循环对象
        System.out.println(group.next());
        System.out.println(group.next());
        System.out.println(group.next());
        //3.执行普通任务
        group.next().submit(() ->{
            try {
                Thread.sleep(1000);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
            System.out.println("qwe");
        });
        //执行定时任务
        group.next().scheduleAtFixedRate(()->{
            System.out.println("123");
        },0, 1, TimeUnit.SECONDS);
    }
}

public class EventLoopServer {
    public static void main(String[] args) {
        EventLoopGroup group = new DefaultEventLoopGroup();
        //1.启动器，负责组装netty组件，启动服务器
        new ServerBootstrap()
                //2.BossEvenLoop，WorkerEvenLoop(selector,thread)
                .group(new NioEventLoopGroup(), new NioEventLoopGroup(2))
                //3.选择服务器的ServerSocketChannel实现
                .channel(NioServerSocketChannel.class)
                //4.boss 负责处理连接 worker(child)负责处理读写，决定worker能执行哪些操作
                .childHandler(
                        //5.channel代表和客户端进行数据读写的通道，Initializer初始化，负责添加别的handler
                        new ChannelInitializer<NioSocketChannel>() {
                            //连接建立后调用initChannel方法
                            @Override
                            protected void initChannel(NioSocketChannel ch) throws Exception {
                                ch.pipeline().addLast("handler1", new ChannelInboundHandlerAdapter(){ //自定义handler
                                    @Override //读事件
                                    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
                                        ByteBuf buf = (ByteBuf)msg;
                                        System.out.println(buf.toString(Charset.defaultCharset()));
                                        //交给下一个handler继续处理
                                        ctx.fireChannelRead(msg);
                                    }
                                }).addLast(group, "handler2", new ChannelInboundHandlerAdapter(){ //自定义handler
                                    @Override //读事件
                                    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
                                        ByteBuf buf = (ByteBuf)msg;
                                        System.out.println(buf.toString(Charset.defaultCharset()));
                                    }
                                });
                            }
                        })
                .bind(8088); //服务器的端口
    }
}

channel:

Future&Promise

jdkFuture：

ublic class TestJdkFuture {
    public static void main(String[] args) throws ExecutionException, InterruptedException {
        ExecutorService service = Executors.newFixedThreadPool(2);
        Future<Integer> future = service.submit(new Callable<Integer>() {
            @Override
            public Integer call() throws Exception {
                Thread.sleep(1000);
                return 66;
            }
        });
        System.out.println(future.get());//线程阻塞获取结果
    }
}

nettyFuture：

public class TestNettyFuture {
    public static void main(String[] args) throws ExecutionException, InterruptedException {
        NioEventLoopGroup group = new NioEventLoopGroup();
        EventLoop loop = group.next();
        Future<Integer> future = loop.submit(() -> {
            System.out.println("执行计算");
            Thread.sleep(1000);
            return 88;
        });
        //同步获取结果
        System.out.println(future.get());
        //异步获取结果
        future.addListener(new GenericFutureListener<Future<? super Integer>>() {
            @Override
            public void operationComplete(Future<? super Integer> future) throws Exception {
                System.out.println("拿到结果：" + future.getNow());
            }
        });
        System.out.println("结束");
    }
}

nettyPromise：

public class TestNettyPromise {
    public static void main(String[] args) throws ExecutionException, InterruptedException {
        EventLoop eventLoop = new NioEventLoopGroup().next();
        DefaultPromise<Integer> promise = new DefaultPromise<>(eventLoop);
        new Thread(()->{
            System.out.println("开始计算");
            try {
                Thread.sleep(1000);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
            //执行计算放到promise中
            promise.setSuccess(30);
        }).start();
        System.out.println(promise.get());
    }
}

Handler&Piepeline：

快速测试类EmbeddedChannel：

public class TestEmbeddedChannel {
    public static void main(String[] args) {
        ChannelInboundHandlerAdapter h1 = new ChannelInboundHandlerAdapter(){
            @Override
            public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
                System.out.println("1");
                //System.out.println(StandardCharsets.UTF_8.decode((ByteBuffer)msg));
                ByteBuf buf = (ByteBuf)msg;
                System.out.println(buf.toString(StandardCharsets.UTF_8));
                super.channelRead(ctx, msg);
            }
        };
        ChannelInboundHandlerAdapter h2 = new ChannelInboundHandlerAdapter(){
            @Override
            public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
                System.out.println("2");
                super.channelRead(ctx, msg);
            }
        };
        ChannelOutboundHandlerAdapter h3 = new ChannelOutboundHandlerAdapter(){
            @Override
            public void write(ChannelHandlerContext ctx, Object msg, ChannelPromise promise) throws Exception {
                System.out.println("3");
                ByteBuf buf = (ByteBuf)msg;
                System.out.println(buf.toString(Charset.defaultCharset()));
                super.write(ctx, msg, promise);
            }
        };
        ChannelOutboundHandlerAdapter h4 = new ChannelOutboundHandlerAdapter(){
            @Override
            public void write(ChannelHandlerContext ctx, Object msg, ChannelPromise promise) throws Exception {
                System.out.println("4");
                super.write(ctx, msg, promise);
            }
        };
        EmbeddedChannel channel = new EmbeddedChannel(h1, h2, h3, h4);
        //模拟入站操作
        channel.writeInbound(ByteBufAllocator.DEFAULT.buffer().writeBytes("fed789".getBytes()));
        //模拟出站操作
        channel.writeOutbound(ByteBufAllocator.DEFAULT.buffer().writeBytes("qwe456".getBytes()));
        //channel.writeInbound(StandardCharsets.UTF_8.encode("fed789456"));
    }
}

Bytebuf：

public class TestByteBuf {
    public static void main(String[] args) {
        ByteBuf buffer = ByteBufAllocator.DEFAULT.buffer();
        System.out.println(buffer);
        StringBuilder stringBuilder = new StringBuilder();
        for (int i = 0; i < 300; i++) {
            stringBuilder.append("f");
        }
        buffer.writeBytes(stringBuilder.toString().getBytes());
        System.out.println(buffer);
    }
}

粘包和半包问题

netty的解码器：定长解码器、行解码器、LTC解码器

LTC解码器：

协议设计与解析

@Data
public abstract class Message implements Serializable {

    /*public static Class<?> getMessageClass(int messageType){
        return null;
    }*/

    private int sequenceId;
    private int messageType;

    public abstract int getMessageType();

    public static final int LoginRequestMessage = 0;
    public static final int LoginResponseMessage = 1;

}

@Slf4j
public class MessageCodec extends ByteToMessageCodec<Message> {

    /**
     * netty自定义协议数据写入
     * @param channelHandlerContext
     * @param message
     * @param byteBuf
     * @throws Exception
     */
    @Override
    protected void encode(ChannelHandlerContext channelHandlerContext, Message message, ByteBuf byteBuf) throws Exception {
        //魔数
        byteBuf.writeBytes(new byte[]{1,2,3,4});
        //版本
        byteBuf.writeByte(1);
        //序列化算法，jdk-0，json-1
        byteBuf.writeByte(0);
        //指令类型
        byteBuf.writeByte(message.getMessageType());
        //请求序号
        byteBuf.writeInt(message.getSequenceId());
        //无意义，对齐填充
        byteBuf.writeInt(0xff);
        //获取正文内容
        ByteArrayOutputStream bos = new ByteArrayOutputStream();
        ObjectOutputStream oos = new ObjectOutputStream(bos);
        oos.writeObject(message);
        byte[] bytes = bos.toByteArray();
        //获取长度
        byteBuf.writeInt(bytes.length);
        //写入内容
        byteBuf.writeBytes(bytes);
    }

    /**
     * netty自定义协议数据读取
     * @param channelHandlerContext
     * @param byteBuf
     * @param list
     * @throws Exception
     */
    @Override
    protected void decode(ChannelHandlerContext channelHandlerContext, ByteBuf byteBuf, List<Object> list) throws Exception {
        int magicNum = byteBuf.readInt();
        byte version = byteBuf.readByte();
        byte serializableType  = byteBuf.readByte();
        byte messageType  = byteBuf.readByte();
        int sequenceId  = byteBuf.readInt();
        byteBuf.readInt();
        int length = byteBuf.readInt();
        byte[] bytes = new byte[length];
        byteBuf.readBytes(bytes,0, length);
        ObjectInputStream ois = new ObjectInputStream(new ByteArrayInputStream(bytes));
        Message message  = (Message) ois.readObject();
        log.debug("{},{},{},{},{},{}", magicNum, version, serializableType, messageType, sequenceId, length);
        log.debug("msg:{}", message);
        list.add(message);
    }

}

不可共享解码类：ByteToMessageCodec

可共享解码类：MessageToMessageCodec，必须要配合LengthFieldBasedFrameDecoder一起使用

CountDownLatch：倒数计时器，可用于两个线程之间通信

连接假死，连接假死检测器：IdleStateHandler，入站出站处理器：ChannelDuplexHandler

对象序列化：

public interface Serializer {
    /**
     * 反序列化
     * @param clazz
     * @param bytes
     * @param <T>
     * @return
     */
    <T> T deserialize(Class<T> clazz, byte[] bytes);

    /**
     * 序列化
     * @param object
     * @param <T>
     * @return
     */
    <T> byte[] serialize(T object);

    enum Algorithm implements Serializer{

        Java{
            @Override
            public <T> T deserialize(Class<T> clazz, byte[] bytes) {
                try {
                    ObjectInputStream ois = new ObjectInputStream(new ByteArrayInputStream(bytes));
                    return (T) ois.readObject();
                }catch (Exception e){
                    throw new RuntimeException("反序列化失败", e);
                }
            }

            @Override
            public <T> byte[] serialize(T object) {
                try {
                    ByteArrayOutputStream bos = new ByteArrayOutputStream();
                    ObjectOutputStream oos = new ObjectOutputStream(bos);
                    oos.writeObject(object);
                    return bos.toByteArray();
                } catch (IOException e) {
                   throw new RuntimeException("序列化失败", e);
                }
            }
        },

        Json {
            @Override
            public <T> T deserialize(Class<T> clazz, byte[] bytes) {
                String json = new String(bytes, StandardCharsets.UTF_8);
                return new Gson().fromJson(json, clazz);
            }

            @Override
            public <T> byte[] serialize(T object) {
                String json = new Gson().toJson(object);
                return json.getBytes(StandardCharsets.UTF_8);
            }
        }
    }
}

半连接队列：tcp3次握手建立连接的时候还没完全建立连接的客户端会先放到这个队列

全连接队列：已经完成tcp3次握手但是服务端还来不及处理的客户端会放到这个队列

ChannelOption.SO_BACKLOG：设置全连接队列的长度

NioEventLoop组成部分：selector（事件监听器），thread（事件处理线程），queue（任务队列）