Netty之EventLoop源码学习

Netty之EventLoop源码学习

@Author Miaoxf
转载请标明出处

netty version 4.1.36

一.主从reactor模型

二.从继承树理解EventLoop

尝试带着几个问题去看代码：

1.eventloop如何实现单线程循环

2.调度和执行队列解耦的意义

1.父类SingleThreadEventExecutor

SingleThreadEventExecutor中封装了一个本地Thread中封装了一个本地Thread

而一个EventLoop在其生命周期中只跟该特定线程绑定。

SingleThreadEventExecutor实现了任务队列功能。

可以通过NioEventLoop实例的execute()添加一个Task，并由EventLoop进行调度。

execute():

EventLoop实例的execute()是其父类方法SingleThreadEventExecutor#execute

//跟踪io.netty.util.concurrent.SingleThreadEventExecutor#execute
//execute()->startThread()->doStartThread()
private void doStartThread() {
    assert thread == null;
        executor.execute(new Runnable() {
            @Override
            public void run() {
                thread = Thread.currentThread();
                if (interrupted) {
                    thread.interrupt();
                }

                boolean success = false;
                updateLastExecutionTime();
                try {
                    SingleThreadEventExecutor.this.run();
                    success = true;
                } catch (Throwable t) {
                    logger.warn("Unexpected exception from an event executor: ", t);
                } finally {
                    //...
                }
            }
        });            
}

跟踪io.netty.channel.nio.NioEventLoop#run中的runAllTasks()/runAllTasks(long timeoutNanos)
protected boolean runAllTasks(long timeoutNanos) {
        fetchFromScheduledTaskQueue();
        Runnable task = pollTask();
        if (task == null) {
            afterRunningAllTasks();
            return false;
        }

        final long deadline = ScheduledFutureTask.nanoTime() + timeoutNanos;
        long runTasks = 0;
        long lastExecutionTime;
        //源码中一般会使用for(;;)而非while(true)
        //编译后会发现for(;;)的指令更简洁
        for (;;) {
            safeExecute(task);

            runTasks ++;

            // Check timeout every 64 tasks because nanoTime() is relatively expensive.
            // XXX: Hard-coded value - will make it configurable if it is really a problem.
            if ((runTasks & 0x3F) == 0) {
                lastExecutionTime = ScheduledFutureTask.nanoTime();
                if (lastExecutionTime >= deadline) {
                    break;
                }
            }

            task = pollTask();
            if (task == null) {
                lastExecutionTime = ScheduledFutureTask.nanoTime();
                break;
            }
        }

        afterRunningAllTasks();
        this.lastExecutionTime = lastExecutionTime;
        return true;
    } 

//runAllTasks()->fetchFromScheduledTaskQueue()
private boolean fetchFromScheduledTaskQueue() {
    long nanoTime = AbstractScheduledEventExecutor.nanoTime();
    //将队列io.netty.util.concurrent.AbstractScheduledEventExecutor#scheduledTaskQueue
    //中的task全部取出并加入队列io.netty.util.concurrent.SingleThreadEventExecutor#taskQueue
    Runnable scheduledTask  = pollScheduledTask(nanoTime);
    while (scheduledTask != null) {
        if (!taskQueue.offer(scheduledTask)) {
            // No space left in the task queue add it back to the scheduledTaskQueue so we pick it up again.
            scheduledTaskQueue().add((ScheduledFutureTask<?>) scheduledTask);
            return false;
        }
        scheduledTask  = pollScheduledTask(nanoTime);
    }
    return true;
}

io.netty.util.concurrent.AbstractScheduledEventExecutor#scheduledTaskQueue这个是调度队列，是一个PriorityQueue，按照task的调度deadline排序。

io.netty.util.concurrent.SingleThreadEventExecutor#taskQueue是执行队列

//runAllTasks()->pollTask()
//从队列io.netty.util.concurrent.SingleThreadEventExecutor#taskQueue中取出task
//包括了上一步post的task
protected Runnable pollTask() {
    assert inEventLoop();
    return pollTaskFrom(taskQueue);
}

//任务执行，最终执行的是io.netty.util.concurrent.SingleThreadEventExecutor#taskQueue中的task
//runAllTasks()->safeExecute()
protected static void safeExecute(Runnable task) {
    try {
        task.run();
    } catch (Throwable t) {
        logger.warn("A task raised an exception. Task: {}", task, t);
    }
}

//该线程循环执行

2.父类AbstractScheduledEventExecutor

AbstractScheduledEventExecutor实现了Schedule功能

可以通过EventLoop的schedule方法来运行一些定时任务。

schedule():

NioEventLoop实例的schedule()在其父类AbstractScheduledEventExecutor中实现。

//io.netty.util.concurrent.AbstractScheduledEventExecutor#schedule(java.lang.Runnable, long, java.util.concurrent.TimeUnit)
//跟踪schedule(Runnable command, long delay, TimeUnit unit)->schedule(final ScheduledFutureTask<V> task)
<V> ScheduledFuture<V> schedule(final ScheduledFutureTask<V> task) {
    if (inEventLoop()) {
        scheduledTaskQueue().add(task);
    } else {
        execute(new Runnable() {
            @Override
            public void run() {
                scheduledTaskQueue().add(task);
            }
        });
    }

    return task;
}

调度队列

是一个PriorityQueue，按照task的调度deadline排序。

public abstract class AbstractScheduledEventExecutor extends AbstractEventExecutor {
	PriorityQueue<ScheduledFutureTask<?>> scheduledTaskQueue;
}

final class ScheduledFutureTask<V> extends PromiseTask<V> implements ScheduledFuture<V>, PriorityQueueNode {
    @Override
    public int compareTo(Delayed o) {
        if (this == o) {
            return 0;
        }

        ScheduledFutureTask<?> that = (ScheduledFutureTask<?>) o;
        long d = deadlineNanos() - that.deadlineNanos();
        if (d < 0) {
            return -1;
        } else if (d > 0) {
            return 1;
        } else if (id < that.id) {
            return -1;
        } else if (id == that.id) {
            throw new Error();
        } else {
            return 1;
        }
    }
    //...
}    

三.从生命周期理解EventLoop

netty version 4.1.51

思考：selector作用？

1.NioEventLoopGroup的初始化

通常new NioEventLoopGroup()时，初始化了eventLoop

//在其父类构造器中
protected MultithreadEventExecutorGroup(int nThreads, Executor executor,
                                        EventExecutorChooserFactory chooserFactory, Object... args) {
	//...
    children = new EventExecutor[nThreads];
    for (int i = 0; i < nThreads; i ++) {
            boolean success = false;
            try {
                //children实际封装了多个eventLoop,每个eventLoop中封装了一个本地线程
                children[i] = newChild(executor, args);
                success = true;
            } catch (Exception e) {
                // TODO: Think about if this is a good exception type
                throw new IllegalStateException("failed to create a child event loop", e);
            } //...
}
    
//初始化eventLoop
//io.netty.channel.nio.NioEventLoopGroup#newChild
protected EventLoop newChild(Executor executor, Object... args) throws Exception {
        EventLoopTaskQueueFactory queueFactory = args.length == 4 ? (EventLoopTaskQueueFactory) args[3] : null;
        return new NioEventLoop(this, executor, (SelectorProvider) args[0],
            ((SelectStrategyFactory) args[1]).newSelectStrategy(), (RejectedExecutionHandler) args[2], queueFactory);
    }    

2.NioEventLoop注册到Channel

eventLoop注册到了io.netty.channel.AbstractChannel#eventLoop中。然后再注册到selector中。

//追踪io.netty.channel.AbstractChannel.AbstractUnsafe#register
public final void register(EventLoop eventLoop, final ChannelPromise promise) {
            ObjectUtil.checkNotNull(eventLoop, "eventLoop");
            if (isRegistered()) {
                promise.setFailure(new IllegalStateException("registered to an event loop already"));
                return;
            }
            if (!isCompatible(eventLoop)) {
                promise.setFailure(
                        new IllegalStateException("incompatible event loop type: " + eventLoop.getClass().getName()));
                return;
            }

            AbstractChannel.this.eventLoop = eventLoop;

            if (eventLoop.inEventLoop()) {
                register0(promise);
            } else {
                try {
                    eventLoop.execute(new Runnable() {
                        @Override
                        public void run() {
                            register0(promise);
                        }
                    });
                } catch (Throwable t) {
                    logger.warn(
                            "Force-closing a channel whose registration task was not accepted by an event loop: {}",
                            AbstractChannel.this, t);
                    closeForcibly();
                    closeFuture.setClosed();
                    safeSetFailure(promise, t);
                }
            }
        }

3.注册时调用了eventLoop.execute(),此时启动了本地线程。

如上代码块，将eventLoop注册到channel之后，由于初始化时，eventloop中的线程还未初始化，所以eventLoop.inEventLoop()=false，执行eventLoop.execute(…)。register0(promise)任务会提交到执行io.netty.util.concurrent.SingleThreadEventExecutor#taskQueue。然后初始化NioEventLoop内部的本地线程并开始执行任务。（register0是第一个任务吗？）

//io.netty.channel.AbstractChannel.AbstractUnsafe#register0->doRegister()
//追踪io.netty.channel.nio.AbstractNioChannel#doRegister
protected void doRegister() throws Exception {
    boolean selected = false;
    for (;;) {
        try {
            selectionKey = javaChannel().register(eventLoop().unwrappedSelector(), 0, this);
            return;
        } catch (CancelledKeyException e) {
            if (!selected) {
                // Force the Selector to select now as the "canceled" SelectionKey may still be
                // cached and not removed because no Select.select(..) operation was called yet.
                eventLoop().selectNow();
                selected = true;
            } else {
                // We forced a select operation on the selector before but the SelectionKey is still cached
                // for whatever reason. JDK bug ?
                throw e;
            }
        }
    }
}

将channel注册给eventLoop中的selector。

//io.netty.util.concurrent.MultithreadEventExecutorGroup#chooser