Netty源码分析NioEventLoopGroup和NioEventLoop ——(一)

机缘巧合之下看了Netty的源码，但是这玩意对我来说使用的频率确实不多，为了加深自己的记忆，也为了以后可以更快的上手，打算写几篇博客从源码角度来梳理下netty的主线， 本篇也是第一篇。

在Netty中它的“线程池”和“线程”分别用NioEventLoopGroup和NioEventLoop来表示，通常我们用下面几行代码来配置服务端和客户端的线程池。

  //server
  ServerBootstrap serverBootstrap = new ServerBootstrap();
  NioEventLoopGroup bossGroup = new NioEventLoopGroup(0, new DefaultThreadFactory("boss"));
  NioEventLoopGroup workGroup = new NioEventLoopGroup(0, new DefaultThreadFactory("worker"));
  serverBootstrap.group(bossGroup, workGroup);
  
  //client 
  Bootstrap bootstrap = new Bootstrap();
  NioEventLoopGroup group = new NioEventLoopGroup();
  bootstrap.group(group);

本篇是第一篇，不会上来就去分析主线，先搞清楚Netty的线程池。
可以看到NioEventLoopGroup的使用非常简单，我们从一个最简单的无参构造开始。

public class NioEventLoopGroup extends MultithreadEventLoopGroup {

    public NioEventLoopGroup() {
        this(0);
    }

    public NioEventLoopGroup(int nThreads) {
        this(nThreads, (Executor)null);
    }
    
    //SelectorProvider.provider()方法获取一个java原生的SelectorProvider对象
    public NioEventLoopGroup(int nThreads, ThreadFactory threadFactory) {
        this(nThreads, threadFactory, SelectorProvider.provider());
    }

    public NioEventLoopGroup(int nThreads, ThreadFactory threadFactory, SelectorProvider selectorProvider) {
        this(nThreads, threadFactory, selectorProvider, DefaultSelectStrategyFactory.INSTANCE);
    }
     
     //调用super的构造
    public NioEventLoopGroup(int nThreads, ThreadFactory threadFactory, SelectorProvider selectorProvider, SelectStrategyFactory selectStrategyFactory) {
        super(nThreads, threadFactory, new Object[]{selectorProvider, selectStrategyFactory, RejectedExecutionHandlers.reject()});
    }
    
	//创建子线程也就是NioEventLoop
    protected EventLoop newChild(Executor executor, Object... args) throws Exception {
        EventLoopTaskQueueFactory queueFactory = args.length == 4 ? (EventLoopTaskQueueFactory)args[3] : null;
        return new NioEventLoop(this, executor, (SelectorProvider)args[0], ((SelectStrategyFactory)args[1]).newSelectStrategy(), (RejectedExecutionHandler)args[2], queueFactory);
    }

NioEventLoopGroup的构造函数中有几个参数需要注意

• nThreads 线程数
• threadFactory 线程工厂
• selectorProvider 用来创建Selector, Selector是Java NIO中的一个组件
• RejectedExecutionHandlers.reject() 拒绝策略，当线程无法处理更多任务的时候会用到

接着调用了父类MultithreadEventLoopGroup的构造。如下：

public abstract class MultithreadEventLoopGroup extends MultithreadEventExecutorGroup implements EventLoopGroup {
 	//...
 	//如果没有配置系统属性，默认的线程数就是cpu核心数*2
    private static final int DEFAULT_EVENT_LOOP_THREADS = Math.max(1, SystemPropertyUtil.getInt("io.netty.eventLoopThreads", NettyRuntime.availableProcessors() * 2));
    
    //由于我们传入的线程数为0 ，默认的线程数就是DEFAULT_EVENT_LOOP_THREADS
    protected MultithreadEventLoopGroup(int nThreads, ThreadFactory threadFactory, Object... args) {
        super(nThreads == 0 ? DEFAULT_EVENT_LOOP_THREADS : nThreads, threadFactory, args);
    }
}

继续追踪super

public abstract class MultithreadEventExecutorGroup extends AbstractEventExecutorGroup {
	//如果threadFactory不为空，就创建ThreadPerTaskExecutor 这个对象是用来真正创建线程Thread的
    protected MultithreadEventExecutorGroup(int nThreads, ThreadFactory threadFactory, Object... args) {
        this(nThreads, (Executor)(threadFactory == null ? null : new ThreadPerTaskExecutor(threadFactory)), args);
    }
    //继续调用其他构造，创建DefaultEventExecutorChooserFactory对象
    protected MultithreadEventExecutorGroup(int nThreads, Executor executor, Object... args) {
        this(nThreads, executor, DefaultEventExecutorChooserFactory.INSTANCE, args);
    }
}

到这里我觉得有必要解释下Netty中的“线程池” NioEventLoopGroup 和“线程” NioEventLoop 其实都只是普通的类。而这里的Executor才是真正用来创建线程的只不过它是给NioEventLoop来使用的，NioEventLoop在执行的时候会通过它创建一个线程，并且持有它，而NioEventLoopGroup是用来创建NioEventLoop，并且管理它的。所以大致上我们可以认为NioEventLoopGroup是线程池，NioEventLoop是线程并没有问题。
首先看下ThreadPerTaskExecutor的源码：

public final class ThreadPerTaskExecutor implements Executor {
    private final ThreadFactory threadFactory;

    public ThreadPerTaskExecutor(ThreadFactory threadFactory) {
        if (threadFactory == null) {
            throw new NullPointerException("threadFactory");
        } else {
            this.threadFactory = threadFactory;
        }
    }
	//非常简单，每来一个任务就创建一个正真的线程Thread
    public void execute(Runnable command) {
        this.threadFactory.newThread(command).start();
    }
}

而DefaultEventExecutorChooserFactory是选择工厂，用来创建选择器，选择器的作用是当任务到达线程池NioEventLoopGroup，NioEventLoopGroup需用通过选择器来选择一个现场NioEventLoop来执行这个任务。
可以看下DefaultEventExecutorChooserFactory的代码如下：

public final class DefaultEventExecutorChooserFactory implements EventExecutorChooserFactory {
    public static final DefaultEventExecutorChooserFactory INSTANCE = new DefaultEventExecutorChooserFactory();

    private DefaultEventExecutorChooserFactory() {
    }

	//创建选择器，当线程池的线程数量是 2^n 就创建PowerOfTwoEventExecutorChooser否则就创建GenericEventExecutorChooser
    public EventExecutorChooser newChooser(EventExecutor[] executors) {
        return (EventExecutorChooser)(isPowerOfTwo(executors.length) ? new DefaultEventExecutorChooserFactory.PowerOfTwoEventExecutorChooser(executors) : new DefaultEventExecutorChooserFactory.GenericEventExecutorChooser(executors));
    }

    private static boolean isPowerOfTwo(int val) {
        return (val & -val) == val;
    }

    private static final class GenericEventExecutorChooser implements EventExecutorChooser {
        private final AtomicInteger idx = new AtomicInteger();
        private final EventExecutor[] executors;

        GenericEventExecutorChooser(EventExecutor[] executors) {
            this.executors = executors;
        }

		//从线程数组中选择一个线程EventExecutor
        public EventExecutor next() {
            return this.executors[Math.abs(this.idx.getAndIncrement() % this.executors.length)];
        }
    }

    private static final class PowerOfTwoEventExecutorChooser implements EventExecutorChooser {
        private final AtomicInteger idx = new AtomicInteger();
        private final EventExecutor[] executors;

        PowerOfTwoEventExecutorChooser(EventExecutor[] executors) {
            this.executors = executors;
        }
		//从线程数组中选择一个线程EventExecutor
        public EventExecutor next() {
            return this.executors[this.idx.getAndIncrement() & this.executors.length - 1];
        }
    }
}

前面分析了MultithreadEventExecutorGroup的两个构造，分别创建了ThreadPerTaskExecutor和DefaultEventExecutorChooserFactory，其实他还有一个构造，也是Netty中线程池的精华。源码如下：

public final class DefaultEventExecutorChooserFactory implements EventExecutorChooserFactory {
    protected MultithreadEventExecutorGroup(int nThreads, Executor executor, EventExecutorChooserFactory chooserFactory, Object... args) {
        this.terminatedChildren = new AtomicInteger();
        this.terminationFuture = new DefaultPromise(GlobalEventExecutor.INSTANCE);
        if (nThreads <= 0) {
            throw new IllegalArgumentException(String.format("nThreads: %d (expected: > 0)", nThreads));
        } else {
        	//如果executor是空就创建一个ThreadPerTaskExecutor
            if (executor == null) {
                executor = new ThreadPerTaskExecutor(this.newDefaultThreadFactory());
            }
			//创建一个线程数组，用来管理所有的“线程”，有点“线程池”那味了
            this.children = new EventExecutor[nThreads];

            int j;
            for(int i = 0; i < nThreads; ++i) {
                boolean success = false;
                boolean var18 = false;

                try {
                    var18 = true;
                    //直接调用newChild创建一个“线程”，并且把executor和args传进去。
                    this.children[i] = this.newChild((Executor)executor, args);
                    success = true;
                    var18 = false;
                } catch (Exception var19) {
                    throw new IllegalStateException("failed to create a child event loop", var19);
                } finally {
                    if (var18) {
                    	//只要有一个失败了success就是false
                        if (!success) {
                            int j;
                            //把所有已经创建的“线程”关闭掉
                            for(j = 0; j < i; ++j) {
                                this.children[j].shutdownGracefully();
                            }

                            for(j = 0; j < i; ++j) {
                                EventExecutor e = this.children[j];

                                try {
                                //等待所有的“线程”关闭
                                    while(!e.isTerminated()) {
                                        e.awaitTermination(2147483647L, TimeUnit.SECONDS);
                                    }
                                } catch (InterruptedException var20) {
                                //设置中断状态
                                    Thread.currentThread().interrupt();
                                    break;
                                }
                            }
                        }

                    }
                }
				//...
            }
			//通过选择器工厂来创建一个选择器，怎么创建选择器，选择器的作用前面已经解释了
            this.chooser = chooserFactory.newChooser(this.children);
            //“线程“终止监听
            FutureListener<Object> terminationListener = new FutureListener<Object>() {
                public void operationComplete(Future<Object> future) throws Exception {
                //当所有的“线程”都终止了，就可以设置线程池也终止了
                    if (MultithreadEventExecutorGroup.this.terminatedChildren.incrementAndGet() == MultithreadEventExecutorGroup.this.children.length) {
                        MultithreadEventExecutorGroup.this.terminationFuture.setSuccess((Object)null);
                    }

                }
            };
            
            EventExecutor[] var24 = this.children;
            j = var24.length;
			//这里给所有的“线程”设置termination监听
            for(int var26 = 0; var26 < j; ++var26) {
                EventExecutor e = var24[var26];
                e.terminationFuture().addListener(terminationListener);
            }
            //。。。
        }
    }
    
   protected abstract EventExecutor newChild(Executor var1, Object... var2) throws Exception;
}

通过newChild方法就可以创建一个“线程池”中的“线程”，接下来我们分析newChild在NioEventLoopGroup中的实现

public class NioEventLoopGroup extends MultithreadEventLoopGroup {
    @Override
    protected EventLoop newChild(Executor executor, Object... args) throws Exception {
        SelectorProvider selectorProvider = (SelectorProvider) args[0];
        SelectStrategyFactory selectStrategyFactory = (SelectStrategyFactory) args[1];
        RejectedExecutionHandler rejectedExecutionHandler = (RejectedExecutionHandler) args[2];
        EventLoopTaskQueueFactory taskQueueFactory = null;
        EventLoopTaskQueueFactory tailTaskQueueFactory = null;

        int argsLength = args.length;
        if (argsLength > 3) {
            taskQueueFactory = (EventLoopTaskQueueFactory) args[3];
        }
        if (argsLength > 4) {
            tailTaskQueueFactory = (EventLoopTaskQueueFactory) args[4];
        }
        return new NioEventLoop(this, executor, selectorProvider,
                selectStrategyFactory.newSelectStrategy(),
                rejectedExecutionHandler, taskQueueFactory, tailTaskQueueFactory);
    }
}

可以看到创建了NioEventLoop，并且把之前创建的一些参数传递进去了。

public final class NioEventLoop extends SingleThreadEventLoop {
	//...
	//newTaskQueue创建了任务队列，我们提交到某个“线程”的任务。最终都会被提交到任务队列中等待被执行
    NioEventLoop(NioEventLoopGroup parent, Executor executor, SelectorProvider selectorProvider,
                 SelectStrategy strategy, RejectedExecutionHandler rejectedExecutionHandler,
                 EventLoopTaskQueueFactory taskQueueFactory, EventLoopTaskQueueFactory tailTaskQueueFactory) {
        super(parent, executor, false, newTaskQueue(taskQueueFactory), newTaskQueue(tailTaskQueueFactory),
                rejectedExecutionHandler);
        this.provider = ObjectUtil.checkNotNull(selectorProvider, "selectorProvider");
        this.selectStrategy = ObjectUtil.checkNotNull(strategy, "selectStrategy");
        //通过selectorProvider创建了Selector
        final SelectorTuple selectorTuple = openSelector();
        this.selector = selectorTuple.selector;
        this.unwrappedSelector = selectorTuple.unwrappedSelector;
    }
}

在NioEventLoop的构造中，又调用了父类的构造了，继续跟进

public abstract class SingleThreadEventLoop extends SingleThreadEventExecutor implements EventLoop {
    protected SingleThreadEventLoop(EventLoopGroup parent, Executor executor,
                                    boolean addTaskWakesUp, Queue<Runnable> taskQueue, Queue<Runnable> tailTaskQueue,
                                    RejectedExecutionHandler rejectedExecutionHandler) {
        super(parent, executor, addTaskWakesUp, taskQueue, rejectedExecutionHandler);
        tailTasks = ObjectUtil.checkNotNull(tailTaskQueue, "tailTaskQueue");
    }
}

public abstract class SingleThreadEventExecutor extends AbstractScheduledEventExecutor implements OrderedEventExecutor
{
    protected SingleThreadEventExecutor(EventExecutorGroup parent, Executor executor,
                                        boolean addTaskWakesUp, Queue<Runnable> taskQueue,
                                        RejectedExecutionHandler rejectedHandler) {
        super(parent);
        this.addTaskWakesUp = addTaskWakesUp;
        this.maxPendingTasks = DEFAULT_MAX_PENDING_EXECUTOR_TASKS;
        this.executor = ThreadExecutorMap.apply(executor, this);
        //任务队列
        this.taskQueue = ObjectUtil.checkNotNull(taskQueue, "taskQueue");
        this.rejectedExecutionHandler = ObjectUtil.checkNotNull(rejectedHandler, "rejectedHandler");
    }
}

到这里线程池和线程创建就分析完成了，但是我们发现其实真正的的线程Thread并没有被创建和执行。我们可以继续看SingleThreadEventExecutor的几个方法。

public abstract class SingleThreadEventExecutor extends AbstractScheduledEventExecutor implements OrderedEventExecutor {
	//真正的线程
	private volatile Thread thread;

    /**
     * Add a task to the task queue, or throws a {@link RejectedExecutionException} if this instance was shutdown
     * before.
     */
    protected void addTask(Runnable task) {
        ObjectUtil.checkNotNull(task, "task");
        if (!offerTask(task)) {
            reject(task);
        }
    }

    final boolean offerTask(Runnable task) {
        if (isShutdown()) {
            reject();
        }
        return taskQueue.offer(task);
    }
    
    protected static void reject() {
        throw new RejectedExecutionException("event executor terminated");
    }
	
	public boolean inEventLoop(Thread thread) {
        return thread == this.thread;
    }
    
	//调用execute去执行任务
    @Override
    public void execute(Runnable task) {
        ObjectUtil.checkNotNull(task, "task");
        //接着调用了下面的execute方法
        execute(task, !(task instanceof LazyRunnable) && wakesUpForTask(task));
    }

    private void execute(Runnable task, boolean immediate) {
    	//判断当前的调用线程是否是NioEventLoop持有的thread
    	//当第一次提交任务的时候会去创建线程
        boolean inEventLoop = inEventLoop();
        //把任务添加到任务队列中
        addTask(task);
        //如果不是，那就启动线程
        if (!inEventLoop) {
            startThread();
            if (isShutdown()) {
                boolean reject = false;
                try {
                    if (removeTask(task)) {
                        reject = true;
                    }
                } catch (UnsupportedOperationException e) {
                    // The task queue does not support removal so the best thing we can do is to just move on and
                    // hope we will be able to pick-up the task before its completely terminated.
                    // In worst case we will log on termination.
                }
                if (reject) {
                    reject();
                }
            }
        }

        if (!addTaskWakesUp && immediate) {
            wakeup(inEventLoop);
        }
    }
    
    private void startThread() {
    	//如果当前NioEventLoop的线程还没有启动过
        if (state == ST_NOT_STARTED) {
        	//更新启动状态
            if (STATE_UPDATER.compareAndSet(this, ST_NOT_STARTED, ST_STARTED)) {
                boolean success = false;
                try {
                	//调用doStartThread去启动线程
                    doStartThread();
                    success = true;
                } finally {
                    if (!success) {
                        STATE_UPDATER.compareAndSet(this, ST_STARTED, ST_NOT_STARTED);
                    }
                }
            }
        }
    }
    private void doStartThread() {
        assert thread == null;
        //这里的executor最终会调用前面的ThreadPerTaskExecutor来创建线程，并且执行Runnable。new Thread(runnable).start()
        executor.execute(new Runnable() {
            @Override
            public void run() {
                thread = Thread.currentThread();
                //...
                try {
                   //调用了run方法
                    SingleThreadEventExecutor.this.run();
                    success = true;
                } finally {
                }
            }
        });
    }
}

NioEventLoop的run方法实现是一个死循环for ，主要是执行所有的任务和对注册到Selector上的事件到来时的处理，但不在本篇的讨论范围，到这里NioEventLoopGroup和NioEventLoop就分析完了。

下一节主要分析Netty服务启动的源码。