NioEventLoop 解析

NioEventLoop 才是实际处理 channel 事件，普通任务，定时任务的类，它会定时轮询select，获取感兴趣的事件并处理，最后处理任务。NioEventLoop 继承了 SingleThreadEventLoop 类，是单线程执行器，一个任务一线程，但是一个 NioEventLoop 可以处理多个不同的channel，这是为什么呢？因为一个 NioEventLoop 拥有一个IO多路复用选择器 Selector，所以它可以处理这个 Selector上被激活的所有channel的任务。

NioEventLoop 会在 NioEventLoopGroup 创建时被创建。

public final class NioEventLoop extends SingleThreadEventLoop {
    private Selector selector;    
    NioEventLoop(NioEventLoopGroup parent, Executor executor, SelectorProvider selectorProvider,
                 SelectStrategy strategy, RejectedExecutionHandler rejectedExecutionHandler) {
        super(parent, executor, false, DEFAULT_MAX_PENDING_TASKS, rejectedExecutionHandler);
        if (selectorProvider == null) {
            throw new NullPointerException("selectorProvider");
        }
        if (strategy == null) {
            throw new NullPointerException("selectStrategy");
        }
        provider = selectorProvider;
        # 开启 多路复用选择器监听Channel； 
        final SelectorTuple selectorTuple = openSelector();
        selector = selectorTuple.selector;
        unwrappedSelector = selectorTuple.unwrappedSelector;
        selectStrategy = strategy;
    }
}

把通道注册到 selector,并设置监听事件(OPS)，传递 NioEventLoop 要执行的任务

public final class NioEventLoop extends SingleThreadEventLoop {
    private Selector selector;
    public void register(final SelectableChannel ch, final int interestOps, final NioTask<?> task) {
        ...
        try {
            # 把通道注册到多路复用选择器里
            ch.register(selector, interestOps, task);
        } catch (Exception e) {
            throw new EventLoopException("failed to register a channel", e);
        }
    }
}

在super() 方法中会做一些初始化，设置 NioEventLoopGroup 为 NioEventLoop 的parent，NioEventLoop会覆写父类的newTaskQueue() 方法。

    @Override
    protected Queue<Runnable> newTaskQueue(int maxPendingTasks) {
        // This event loop never calls takeTask()
        return maxPendingTasks == Integer.MAX_VALUE ? PlatformDependent.<Runnable>newMpscQueue()
                                                    : PlatformDependent.<Runnable>newMpscQueue(maxPendingTasks);
    }

newMpscQueue 是多生产者单消费者队列。

当有一个任务要被执行时，它会先去执行 SingleThreadEventExecutor#execute()

@Override
    public void execute(Runnable task) {
        if (task == null) {
            throw new NullPointerException("task");
        }
        boolean inEventLoop = inEventLoop();
        addTask(task);
        if (!inEventLoop) {
            startThread();
            if (isShutdown() && removeTask(task)) {
                reject();
            }
        }
        if (!addTaskWakesUp && wakesUpForTask(task)) {
            wakeup(inEventLoop);
        }
    }

并最终执行SingleThreadEventExecutor.this.run() ,这里的run()就是NioEventLoop#run()方法。一般情况下都是走SelectStrategy.SELECT 分支。

    @Override
    protected void run() {
        for (;;) {
            try {
                switch (selectStrategy.calculateStrategy(selectNowSupplier, hasTasks())) {
                    case SelectStrategy.CONTINUE:
                        continue;
                    case SelectStrategy.SELECT:
                        select(wakenUp.getAndSet(false));
                        // 这是运行多线程执行的，如果A线程执行了上一行代码后wakenUp为false，但另一个线程可能执行了wakeup()，它会导致运行到此处的A线程唤醒selector
                        if (wakenUp.get()) {
                            selector.wakeup();
                        }
                    default:
                }
                cancelledKeys = 0;
                needsToSelectAgain = false;
                final int ioRatio = this.ioRatio;
                if (ioRatio == 100) {
                    try {
                        processSelectedKeys();
                    } finally {
                        // Ensure we always run tasks.
                        runAllTasks();
                    }
                } else {
                    final long ioStartTime = System.nanoTime();
                    try {
                        processSelectedKeys();
                    } finally {
                        // Ensure we always run tasks.
                        final long ioTime = System.nanoTime() - ioStartTime;
                        runAllTasks(ioTime * (100 - ioRatio) / ioRatio);
                    }
                }
            } catch (Throwable t) {
                handleLoopException(t);
            }
            try {
                if (isShuttingDown()) {
                    closeAll();
                    if (confirmShutdown()) {
                        return;
                    }
                }
            } catch (Throwable t) {
                handleLoopException(t);
            }
        }
    }

上面的select(false) 会执行

    private void select(boolean oldWakenUp) throws IOException {
        Selector selector = this.selector;
        try {
            int selectCnt = 0;
            long currentTimeNanos = System.nanoTime();
            long selectDeadLineNanos = currentTimeNanos + delayNanos(currentTimeNanos);
            for (;;) {
                long timeoutMillis = (selectDeadLineNanos - currentTimeNanos + 500000L) / 1000000L;
                // debug时发现跳过不执行
                if (timeoutMillis <= 0) {
                    if (selectCnt == 0) {
                        selector.selectNow();
                        selectCnt = 1;
                    }
                    break;
                }
                // debug时发现跳过不执行
                if (hasTasks() && wakenUp.compareAndSet(false, true)) {
                    selector.selectNow();
                    selectCnt = 1;
                    break;
                }
                int selectedKeys = selector.select(timeoutMillis);
                selectCnt ++;
                if (selectedKeys != 0 || oldWakenUp || wakenUp.get() || hasTasks() || hasScheduledTasks()) {
                    // - Selected something,
                    // - waken up by user, or
                    // - the task queue has a pending task.
                    // - a scheduled task is ready for processing
                    break;
                }
                if (Thread.interrupted()) {
                    selectCnt = 1;
                    break;
                }
                long time = System.nanoTime();
                if (time - TimeUnit.MILLISECONDS.toNanos(timeoutMillis) >= currentTimeNanos) {
                    // timeoutMillis elapsed without anything selected.
                    selectCnt = 1;
                } else if (SELECTOR_AUTO_REBUILD_THRESHOLD > 0 &&
                        selectCnt >= SELECTOR_AUTO_REBUILD_THRESHOLD) {
                    // The selector returned prematurely many times in a row.
                    // Rebuild the selector to work around the problem.
                    rebuildSelector();
                    selector = this.selector;
                    // Select again to populate selectedKeys.
                    selector.selectNow();
                    selectCnt = 1;
                    break;
                }
                currentTimeNanos = time;
            }
        }  
        }
    }

wakeup()方法会唤醒 selector ，唤醒阻塞在selector.select()上的线程，因为执行Selector#select(long timeout) 方法（阻塞等待有 Channel 感兴趣的 IO 事件，或者超时）会导致很多线程阻塞，NioEventLoop 的线程也可能会阻塞，inEventLoop() 返回false。通过CAS的方式可以保证只唤醒一次。

@Override
protected void wakeup(boolean inEventLoop) {
    if (!inEventLoop && wakenUp.compareAndSet(false, true)) { 
        selector.wakeup();
    }
}

这里Netty会去解决epoll空轮询的bug，当 selectCnt 超过阈值就去重建。重建的时候就是把oldSelector 的 SelectionKey的数据转移到新Selector ，Channel 重新注册到到新Selector里。

SelectionKey 是一个SelectableChannel向 Selector注册时的令牌，它在注册的时候被创建。SelectionKeyImpl就持有SelectableChannel和Selector的引用，当一个SelectionKey被cancel时，它不会立即从Selector中删除，而是加入到一个失效列表里。

void cancel(SelectionKey k) {                       
        synchronized (cancelledKeys) {
            cancelledKeys.add(k);
        }
    }

// 重建Selector
private void rebuildSelector0() {
        final Selector oldSelector = selector;
        final SelectorTuple newSelectorTuple;
        if (oldSelector == null) {
            return;
        }
        try {
            newSelectorTuple = openSelector();
        } catch (Exception e) {
            return;
        }
        // Register all channels to the new Selector.
        int nChannels = 0;
        for (SelectionKey key: oldSelector.keys()) {
            Object a = key.attachment();
            try {
                if (!key.isValid() || key.channel().keyFor(newSelectorTuple.unwrappedSelector) != null) {
                    continue;
                }

                int interestOps = key.interestOps();
                key.cancel();
                SelectionKey newKey = key.channel().register(newSelectorTuple.unwrappedSelector, interestOps, a);
                if (a instanceof AbstractNioChannel) {
                    // Update SelectionKey
                    ((AbstractNioChannel) a).selectionKey = newKey;
                }
                nChannels ++;
            } catch (Exception e) {
                if (a instanceof AbstractNioChannel) {
                    AbstractNioChannel ch = (AbstractNioChannel) a;
                    ch.unsafe().close(ch.unsafe().voidPromise());
                } else {
                    NioTask<SelectableChannel> task = (NioTask<SelectableChannel>) a;
                    invokeChannelUnregistered(task, key, e);
                }
            }
        }
        selector = newSelectorTuple.selector;
        unwrappedSelector = newSelectorTuple.unwrappedSelector;
        try {
            oldSelector.close();
        } catch (Throwable t) {
        }
    }

NioEventLoop在处理IO事件时，它会select 所有就绪事件的SelectionKey，然后依次处理。

public class SelectionKeyImpl extends AbstractSelectionKey {
    final SelChImpl channel;
    public final SelectorImpl selector;
    private int index;
    // 这个SelectedKey 感兴趣的事件
    private volatile int interestOps;
    // 实际到来的事件
    private int readyOps;
}

因为readInterestOp为1，显然它是一个读事件。参数a是NioSocketChannel 是产生事件的channel。

private void processSelectedKey(SelectionKey k, AbstractNioChannel ch) {
        final AbstractNioChannel.NioUnsafe unsafe = ch.unsafe();
        if (!k.isValid()) {
            final EventLoop eventLoop;
            try {
                eventLoop = ch.eventLoop();
            } catch (Throwable ignored) {
                return;
            }
            if (eventLoop != this || eventLoop == null) {
                return;
            }
            // close the channel if the key is not valid anymore
            unsafe.close(unsafe.voidPromise());
            return;
        }

        try {
            int readyOps = k.readyOps();
            if ((readyOps & SelectionKey.OP_CONNECT) != 0) {
                // remove OP_CONNECT as otherwise Selector.select(..) will always return without blocking
                int ops = k.interestOps();
                ops &= ~SelectionKey.OP_CONNECT;
                k.interestOps(ops);

                unsafe.finishConnect();
            }
            // Process OP_WRITE first as we may be able to write some queued buffers and so free memory.
            if ((readyOps & SelectionKey.OP_WRITE) != 0) {
                ch.unsafe().forceFlush();
            }
            if ((readyOps & (SelectionKey.OP_READ | SelectionKey.OP_ACCEPT)) != 0 || readyOps == 0) {
                unsafe.read();
            }
        } catch (CancelledKeyException ignored) {
            unsafe.close(unsafe.voidPromise());
        }
    }

这时候，连接已经建立。它会进入如下方法去接收数据了。再往下执行就是channel的部分了。

if ((readyOps & (SelectionKey.OP_READ | SelectionKey.OP_ACCEPT)) != 0 || readyOps == 0) {
                unsafe.read();
            }

处理完SelectedKey后，它就会执行 runAllTasks() 去处理所有的任务了。默认情况下ioRatio 不会等于100，所以它会执行带有超时的runAllTasks(long timeoutNanos)。这些任务里就包含了普通任务和定时任务。

    protected boolean runAllTasks(long timeoutNanos) {
        fetchFromScheduledTaskQueue();
        Runnable task = pollTask();
        if (task == null) {
            afterRunningAllTasks();
            return false;
        }
        final long deadline = ScheduledFutureTask.nanoTime() + timeoutNanos;
        long runTasks = 0;
        long lastExecutionTime;
        for (;;) {
            safeExecute(task);
            runTasks ++;
            // 没执行64个任务就看一下是否超时，因为 nanoTime() 是相对费时的操作
            // 64是硬编码，如果我们有需要也可以很容易变为可配置的参数
            if ((runTasks & 0x3F) == 0) {
                lastExecutionTime = ScheduledFutureTask.nanoTime();
                if (lastExecutionTime >= deadline) {
                    break;
                }
            }
            task = pollTask();
            if (task == null) {
                lastExecutionTime = ScheduledFutureTask.nanoTime();
                break;
            }
        }

        afterRunningAllTasks();
        this.lastExecutionTime = lastExecutionTime;
        return true;
    }

NioEventLoop还是挺复杂的，它负责的事情很多。其中NioEventLoop和java NIO里的很多概念是相通的。debug的过程中遇到一些类，它们的作用是什么样的，可能在执行流程的代码里想不太明白，这时候要看注释，或是直接百度它。

未完待续。。。