Netty服务端源码阅读笔记（五）AdaptiveRecvByteBufAllocator

在读服务端NioServerSocketChannel和客户端NioSocketChannel的内部unsafe类读写数据源码时，发现有RecvByteBufAllocator对象来辅助操作，另写一篇来分析分析

其实客户端读数据才是难点重点，但是我们先分析服务端channel读取数据方法，比较简单好了解

目录

服务端NioServerSocketChannel.NioMessageUnsafe

1、new AdaptiveRecvByteBufAllocator ()

2、 allocHandler.reset

3、allocHandler.incMessagesRead

4、 allocHandler.continueReading

客户端NioSocketChannel.NioSocketChannelUnsafe

1、final ByteBufAllocator allocator = config.getAllocator()  

2、allocHandle.reset(config)

3、byteBuf = allocHandle.allocate(allocator)

4、allocHandle.lastBytesRead(doReadBytes(byteBuf));

5、allocHandle.incMessagesRead(1)

6、allocHandle.continueReading()

7、allocHandle.readComplete()

---

 

 

服务端NioServerSocketChannel.NioMessageUnsafe

代码位置：io.netty.channel.nio.AbstractNioMessageChannel.NioMessageUnsafe#read

       public void read() {
            assert eventLoop().inEventLoop();
            final ChannelConfig config = config();
            final ChannelPipeline pipeline = pipeline();

            // 获取RecvByteBufAllocator中的Handler对象
            final RecvByteBufAllocator.Handle allocHandle = unsafe().recvBufAllocHandle();

             // 初始化
            allocHandle.reset(config);

            boolean closed = false;
            Throwable exception = null;
            try {
                try {
                    do {// 读入客户端连接SocketChannel，accept()
                        int localRead = doReadMessages(readBuf);
                        if (localRead == 0) {
                            break;
                        }
                        if (localRead < 0) {
                            closed = true;
                            break;
                        }

                        allocHandle.incMessagesRead(localRead);
                    } while (allocHandle.continueReading());
                } catch (Throwable t) {
                    exception = t;
                }

                int size = readBuf.size();
                for (int i = 0; i < size; i ++) {
                    readPending = false;
                   // 调用服务端channelRead，将客户端socketChannel注册到childGroup
                    pipeline.fireChannelRead(readBuf.get(i));
                }
                readBuf.clear();

           //省略部分代码
}

        @Override
        public RecvByteBufAllocator.Handle recvBufAllocHandle() {
            if (recvHandle == null) {
                recvHandle = config().getRecvByteBufAllocator().newHandle();
            }
            return recvHandle;
        }

 

 处理流程大致一看：获取allocHandler对象，首先reset()方法初始化，然后do while循环，每次doReadMessages(readBuf)读取消息后，调用 incMessagesRead更新消息读取情况，再调用continueReading判断是否可以继续读取。来看这几个方法和RecvByteBufAllocator对象都是什么

先找到RecvByteBufAllocator对象初始化的地方，来看new NioServerSocketChannel初始化时的代码，初始化了一个Config对象

   public NioServerSocketChannel() {
        this(newSocket(DEFAULT_SELECTOR_PROVIDER));
    }

    public NioServerSocketChannel(SelectorProvider provider) {
        this(newSocket(provider));
    }

    public NioServerSocketChannel(ServerSocketChannel channel) {
        // 封装原生channel，设置关注事件为ACCEPT
        super(null, channel, SelectionKey.OP_ACCEPT);
        // 初始化channel的配置对象
        config = new NioServerSocketChannelConfig(this, javaChannel().socket());
    }

 config对象中初始化了AdaptiveRecvByteBufAllocator对象

    private NioServerSocketChannelConfig(NioServerSocketChannel channel, ServerSocket javaSocket) {
            super(channel, javaSocket);
        }

    // 往父类找，这个到了，初始化了一个RecvByteBufAllocator对象
    public DefaultChannelConfig(Channel channel) {
        this(channel, new AdaptiveRecvByteBufAllocator());
    }

    protected DefaultChannelConfig(Channel channel, RecvByteBufAllocator allocator) {
        // channel.metedata()为 ChannelMetadata METADATA = new ChannelMetadata(false, 16);
        setRecvByteBufAllocator(allocator, channel.metadata());
        this.channel = channel;
    }

  private void setRecvByteBufAllocator(RecvByteBufAllocator allocator, ChannelMetadata metadata) {
        if (allocator instanceof MaxMessagesRecvByteBufAllocator) {
            // 设置AdaptiveRecvByteBufAllocator默认的每次读取的最大消息数量，服务端channel默认16
            ((MaxMessagesRecvByteBufAllocator) 
allocator).maxMessagesPerRead(metadata.defaultMaxMessagesPerRead());
        } else if (allocator == null) {
            throw new NullPointerException("allocator");
        }
        setRecvByteBufAllocator(allocator);
    }

1、new AdaptiveRecvByteBufAllocator ()

Adaptive适应的，可调整的，接收消息时，会有个默认大小的缓冲区ByteBuf，如果消息过大，这个对象可以逐步扩容缓冲区

public class AdaptiveRecvByteBufAllocator extends DefaultMaxMessagesRecvByteBufAllocator {

    static final int DEFAULT_MINIMUM = 64;
    static final int DEFAULT_INITIAL = 1024;
    static final int DEFAULT_MAXIMUM = 65536;

  // 存了各种2的幂值，表示读取数据时的缓冲buf大小，可以动态调整Adaptive，默认1024,最大65536
// [16, 32, 64, ......, 1024, 2048, ....., 1073741824]
  private static final int[] SIZE_TABLE;

    static {
        List<Integer> sizeTable = new ArrayList<Integer>();
        for (int i = 16; i < 512; i += 16) {
            sizeTable.add(i);
        }

        for (int i = 512; i > 0; i <<= 1) {
            sizeTable.add(i);
        }

        SIZE_TABLE = new int[sizeTable.size()];
        for (int i = 0; i < SIZE_TABLE.length; i ++) {
            SIZE_TABLE[i] = sizeTable.get(i);
        }
    }
 
  public AdaptiveRecvByteBufAllocator() {
        this(DEFAULT_MINIMUM, DEFAULT_INITIAL, DEFAULT_MAXIMUM);
    }

    /**
     * Creates a new predictor with the specified parameters.
     *
     * @param minimum  the inclusive lower bound of the expected buffer size
     * @param initial  the initial buffer size when no feed back was received
     * @param maximum  the inclusive upper bound of the expected buffer size
     */
    public AdaptiveRecvByteBufAllocator(int minimum, int initial, int maximum) {
        checkPositive(minimum, "minimum");
        if (initial < minimum) {
            throw new IllegalArgumentException("initial: " + initial);
        }
        if (maximum < initial) {
            throw new IllegalArgumentException("maximum: " + maximum);
        }

        int minIndex = getSizeTableIndex(minimum);
        if (SIZE_TABLE[minIndex] < minimum) {
            this.minIndex = minIndex + 1;
        } else {
            this.minIndex = minIndex;
        }

        int maxIndex = getSizeTableIndex(maximum);
        if (SIZE_TABLE[maxIndex] > maximum) {
            this.maxIndex = maxIndex - 1;
        } else {
            this.maxIndex = maxIndex;
        }

        this.initial = initial;
    }
}

到开头截图里的newHandler()方法

@SuppressWarnings("deprecation")
@Override
 // io.netty.channel.AdaptiveRecvByteBufAllocator#newHandle
public Handle newHandle() {
    return new HandleImpl(minIndex, maxIndex, initial);
}

    // AdaptiveRecvByteBufAllocator的内部类
    private final class HandleImpl extends MaxMessageHandle {
        private final int minIndex;
        private final int maxIndex;
        private int index;
        private int nextReceiveBufferSize;
        private boolean decreaseNow;

        HandleImpl(int minIndex, int maxIndex, int initial) {
            this.minIndex = minIndex;
            this.maxIndex = maxIndex;

           // 这里获取刚才那个二的幂数组下标和值
           // 用来表示当前接收缓冲区的大小，每次接受完信息会判断是否需要扩容
            index = getSizeTableIndex(initial);
            nextReceiveBufferSize = SIZE_TABLE[index];
        }
    }

2、 allocHandler.reset

        @Override
        public void reset(ChannelConfig config) {
            this.config = config;
            // 上边看过的，默认16
            maxMessagePerRead = maxMessagesPerRead();
            // 接收到的消息数totalMessages /字节数totalBytesRead  初始为0
            totalMessages = totalBytesRead = 0;
        }

3、allocHandler.incMessagesRead

        @Override
        public final void incMessagesRead(int amt) {
            // 很简单，对于服务端channel来说每次doReadMessage如果获取到连接，返回1，这里加一
            // 如果没获取到连接直接break了，不走这
            totalMessages += amt;
        }

4、 allocHandler.continueReading

        @Override
        public boolean continueReading() {
            return continueReading(defaultMaybeMoreSupplier);
        }

        @Override
        public boolean continueReading(UncheckedBooleanSupplier maybeMoreDataSupplier) {
            // 服务端channel这里永远false，每次读一个客户端连接完事，因为totalBytesRead没有用到，永远为0
            // 其他的判断在客户端read的时候会用到
            return config.isAutoRead() &&
                   (!respectMaybeMoreData || maybeMoreDataSupplier.get()) &&
                   totalMessages < maxMessagePerRead &&
                   totalBytesRead > 0;
        }

客户端NioSocketChannel.NioSocketChannelUnsafe

继承自NioByteUnsafe，代码位置：io.netty.channel.nio.AbstractNioByteChannel.NioByteUnsafe

    protected class NioByteUnsafe extends AbstractNioUnsafe {
        @Override
        public final void read() {
            final ChannelConfig config = config();
            if (shouldBreakReadReady(config)) {
                clearReadPending();
                return;
            }
            final ChannelPipeline pipeline = pipeline();
            // 用来分配ByteBuffer缓冲区的对象，另外详解
            final ByteBufAllocator allocator = config.getAllocator();

            // 同服务端 AdaptiveRecvByteBufAllocator 对象
            final RecvByteBufAllocator.Handle allocHandle = recvBufAllocHandle();

            // 开始读消息，置各种标志位为0，如totalMessages/totalBytesRead，顾名思义
            allocHandle.reset(config);

            ByteBuf byteBuf = null;
            boolean close = false;
            try {
                do {
                    // 分配缓冲区空间ByteBuf ，和Nio中的ByteBuffer不一样，类似，后边详看
                    byteBuf = allocHandle.allocate(allocator);

                    // doReadBytes每次将从channel中读取信息到byteBuf，返回值为读了多少字节，int型
                    // lastBytesRead，保存本次读了多少字节(lastBytesRead)，并更新总读入字节数(totalBytesRead)
                    allocHandle.lastBytesRead(doReadBytes(byteBuf));

                    // 如果没有读到东西，释放缓冲区，跳出循环
                    if (allocHandle.lastBytesRead() <= 0) {
                        // nothing was read. release the buffer.
                        byteBuf.release();
                        byteBuf = null;
                        close = allocHandle.lastBytesRead() < 0;
                        if (close) {
                            // There is nothing left to read as we received an EOF.
                            readPending = false;
                        }
                        break;
                    }

                    // 增加读次数，RecvByteBufAllocator.allocHandle 有最大读次数限制
                    allocHandle.incMessagesRead(1);
                    readPending = false;
                    
                    // 调用handler.channelRead
                    pipeline.fireChannelRead(byteBuf);
                    byteBuf = null;
                    
                    // totalMessages < maxMessagePerRead && totalBytesRead > 0
                    // 没到最大读取次数 && 总读入数大于0
                } while (allocHandle.continueReading());

                allocHandle.readComplete();

            // 省略了部分代码
    }
}

1、final ByteBufAllocator allocator = config.getAllocator()  

ByteBufAllocator  new缓冲区用的，暂时不管细节，另外篇，传送门：待续。。。

2、allocHandle.reset(config)

不看了，之前看过了，初始化总数为0

3、byteBuf = allocHandle.allocate(allocator)

分配缓冲区，分两缓冲区，堆里和内存里，本文不深入

// AdaptiveRecvByteBufAllocator 的父类
public abstract class DefaultMaxMessagesRecvByteBufAllocator implements MaxMessagesRecvByteBufAllocator { 
 public ByteBuf allocate(ByteBufAllocator alloc) {
            return alloc.ioBuffer(guess());
  }
}

// AdaptiveRecvByteBufAllocator 的内部类
 private final class HandleImpl extends MaxMessageHandle {
        // 上边看过的，nextReceiveBufferSize表示当前分配的缓冲区大小
        @Override
        public int guess() {
            return nextReceiveBufferSize;
        }
}

//io.netty.buffer.AbstractByteBufAllocator#ioBuffer(int)
    @Override
    public ByteBuf ioBuffer(int initialCapacity) {
        if (PlatformDependent.hasUnsafe()) {
            return directBuffer(initialCapacity);
        }
        return heapBuffer(initialCapacity);
    }

4、allocHandle.lastBytesRead(doReadBytes(byteBuf));

4.1 doReadBytes(byteBuf)

    protected int doReadBytes(ByteBuf byteBuf) throws Exception {
        final RecvByteBufAllocator.Handle allocHandle = unsafe().recvBufAllocHandle();
        // 设置allocHandler最大能读多少东西
        // byteBuf.writableBytes()，当前可用空间
        allocHandle.attemptedBytesRead(byteBuf.writableBytes());

        // 从channel读到byteBuf中，返回读入字节数
        return byteBuf.writeBytes(javaChannel(), allocHandle.attemptedBytesRead());
    }

//io.netty.channel.DefaultMaxMessagesRecvByteBufAllocator.MaxMessageHandle#attemptedBytesRead(int)
       @Override
        public void attemptedBytesRead(int bytes) {
            attemptedBytesRead = bytes;
        }

4.2 allocHandle.lastBytesRead

数据量大则扩容

  //  io.netty.channel.AdaptiveRecvByteBufAllocator.HandleImpl#lastBytesRead   
     public void lastBytesRead(int bytes) {
  
            // 如果读到的字节数等于可以读到的最大字节数，即读到的数据大小把缓冲区占满了
            // 进行record，扩容操作
            if (bytes == attemptedBytesRead()) {
                record(bytes);
            }
            // 父类是保存本次读取的字节数，更新总读入字节数
            super.lastBytesRead(bytes);
        }

       //io.netty.channel.DefaultMaxMessagesRecvByteBufAllocator.MaxMessageHandle#lastBytesRead(int)
        @Override
        public void lastBytesRead(int bytes) {
            lastBytesRead = bytes;
            if (bytes > 0) {
                totalBytesRead += bytes;
            }
        }

//io.netty.channel.AdaptiveRecvByteBufAllocator.HandleImpl#record
      private void record(int actualReadBytes) {
            // 如果总读入的数量比当前缓冲区大小 小两个数量级，那该缩小每次分配的缓冲区大小
            if (actualReadBytes <= SIZE_TABLE[max(0, index - INDEX_DECREMENT - 1)]) {
                if (decreaseNow) {
                    // 如果当前不是刚刚被缩小/放大
                    // 大小减一半
                    index = max(index - INDEX_DECREMENT, minIndex);
                    nextReceiveBufferSize = SIZE_TABLE[index];

                    // decreaseNow 为false，给一段过渡期
                    decreaseNow = false;
                } else {
                    //过渡期
                    decreaseNow = true;
                }
            } else if (actualReadBytes >= nextReceiveBufferSize) {
                // 如果缓冲区小于总读入数，那该扩容了，每次扩2^4个级
                index = min(index + INDEX_INCREMENT, maxIndex);
                nextReceiveBufferSize = SIZE_TABLE[index];
                decreaseNow = false;
            }
        }

    private static final int INDEX_INCREMENT = 4;// 每次大小扩2*4
    private static final int INDEX_DECREMENT = 1;// 每次缩小一半

5、allocHandle.incMessagesRead(1)

加1总数

        @Override
        public final void incMessagesRead(int amt) {
            totalMessages += amt;
        }

6、allocHandle.continueReading()

判断是否继续do while循环

       private final UncheckedBooleanSupplier defaultMaybeMoreSupplier = new UncheckedBooleanSupplier() {
            @Override
            public boolean get() {
                // 每次doReadMessage会更新attemptedBytesRead为当前byteBuf的可用空间
                // 这里判断如果 上一次读入的字节数等于最大可读数，认为可能还会有未读完的数据
                // do while时，会再次循环继续读
                return attemptedBytesRead == lastBytesRead;
            }
        };

        @Override
        public boolean continueReading() {
            return continueReading(defaultMaybeMoreSupplier);
        }

        @Override
        public boolean continueReading(UncheckedBooleanSupplier maybeMoreDataSupplier) {
            return config.isAutoRead() &&
                   // 可能还有未读完的数据 && 总的读取次数< 每次最大读取次数（默认16次） && 上次读到数了
                   (!respectMaybeMoreData || maybeMoreDataSupplier.get()) &&
                   totalMessages < maxMessagePerRead &&
                   totalBytesRead > 0;
        }

7、allocHandle.readComplete()

io.netty.channel.AdaptiveRecvByteBufAllocator.HandleImpl#readComplete

        @Override
        public void readComplete() {
            // 再判断是否需要扩容和缩小缓冲区，上边看过
            record(totalBytesRead());
        }

        protected final int totalBytesRead() {
            // 获取本次读到的总数
            return totalBytesRead < 0 ? Integer.MAX_VALUE : totalBytesRead;
        }

 

留了个ByteBuf的分配 。下篇  nio的byteBuffer和netty的byteBuf