Netty源码分析:服务端接收请求建立连接过程、bossGroup移交socketChannel到workerGroup

概述

  1. Netty抽象出两组线程池:bossGroup(专门负责接收客户端的连接)和workerGroup(专门负责处理连接)
  2. 通常情况下bossGroup和workerGroup都是NioEventLoopGroup
  3. NioEventLoopGroup相当于一个循环组,这个组中含有多个事件循环,也就是NioEventLoop(内部使用一个一直阻塞的循环代码块——for (;;))
  4. NioEventLoop表示一个不断循环的执行处理任务的线程,每个NioEventLoop都有一个selector,用于监听绑定在其上的socket网络通讯
  5. 每个boosGroup内的NioEventLoop循环执行的步骤分为三步:
    1. 轮询accept事件
    2. 处理accept事件,于client建立连接,生成NioSocketChannel,并将其注册到某个workerGroup中的NioEventLoop的selector上
    3. 处理任务队列的任务,继续执行以上步骤,即runAllTasks
  6. 每个workGroup内的NioEventLoop循环执行的步骤分为三步:
    1. 轮询read和write等事件
    2. 处理I/O事件,即read和write等事件,对应NioSocketChannel处理
    3. 处理任务队列的任务,继续执行以上步骤,即runAllTasks

跟踪代码分析

初始化

ServerBootstrap:

serverBootstrap.group(bossGroup, workerGroup)

AbstractBootstrap:

    volatile EventLoopGroup group;

    /**
     * The {@link EventLoopGroup} which is used to handle all the events for the to-be-created
     * {@link Channel}
     */
    public B group(EventLoopGroup group) {
        ObjectUtil.checkNotNull(group, "group");
        if (this.group != null) {
            throw new IllegalStateException("group set already");
        }
        this.group = group;
        return self();
    }

通过ServerBootstrap.group()方法,bossGroup对象最终被保存在了ServerBootstrap父类AbstractBootstrap的group成员变量上

启动过程

AbstractBootstrap:

    final ChannelFuture initAndRegister() {
        Channel channel = null;
        try {
            channel = channelFactory.newChannel();
            init(channel);
        } catch (Throwable t) {
            if (channel != null) {
                // channel can be null if newChannel crashed (eg SocketException("too many open files"))
                channel.unsafe().closeForcibly();
                // as the Channel is not registered yet we need to force the usage of the GlobalEventExecutor
                return new DefaultChannelPromise(channel, GlobalEventExecutor.INSTANCE).setFailure(t);
            }
            // as the Channel is not registered yet we need to force the usage of the GlobalEventExecutor
            return new DefaultChannelPromise(new FailedChannel(), GlobalEventExecutor.INSTANCE).setFailure(t);
        }

        ChannelFuture regFuture = config().group().register(channel);
        if (regFuture.cause() != null) {
            if (channel.isRegistered()) {
                channel.close();
            } else {
                channel.unsafe().closeForcibly();
            }
        }

        // If we are here and the promise is not failed, it's one of the following cases:
        // 1) If we attempted registration from the event loop, the registration has been completed at this point.
        //    i.e. It's safe to attempt bind() or connect() now because the channel has been registered.
        // 2) If we attempted registration from the other thread, the registration request has been successfully
        //    added to the event loop's task queue for later execution.
        //    i.e. It's safe to attempt bind() or connect() now:
        //         because bind() or connect() will be executed *after* the scheduled registration task is executed
        //         because register(), bind(), and connect() are all bound to the same thread.

        return regFuture;
    }

在调用serverBootstrap.bind()方法启动服务的时候:

 

 

  1. 通过config()获取到初始化ServerBootstrap时候初始化的ServerBootstrapConfig对象,ServerBootstrapConfig作为成员变量在ServerBootstrap中被初始化的时候又“持有了”当前的ServerBootstrap对象
    private final ServerBootstrapConfig config = new ServerBootstrapConfig(this);

    并将该ServerBootstrap对象传递到了父类构造方法中,在AbstractBootstrapConfig中赋值给了其成员变量bootstrap

        protected final B bootstrap;
    
        protected AbstractBootstrapConfig(B bootstrap) {
            this.bootstrap = ObjectUtil.checkNotNull(bootstrap, "bootstrap");
        }

    所以可以通过config().group()获取到bossGroup对象

  2. bossGroup.register(Channel)调用的是父类的MultithreadEventLoopGroup.register()

        @Override
        public ChannelFuture register(Channel channel) {
            return next().register(channel);
        }

    逐级往下跟进代码找到SingleThreadEventLoop:

        @Override
        public ChannelFuture register(Channel channel) {
            return register(new DefaultChannelPromise(channel, this));
        }
    
        @Override
        public ChannelFuture register(final ChannelPromise promise) {
            ObjectUtil.checkNotNull(promise, "promise");
            promise.channel().unsafe().register(this, promise);
            return promise;
        }
    
        

    继续跟进到AbstractChannel:

        @Override
            public final void register(EventLoop eventLoop, final ChannelPromise promise) {
                ObjectUtil.checkNotNull(eventLoop, "eventLoop");
                if (isRegistered()) {
                    promise.setFailure(new IllegalStateException("registered to an event loop already"));
                    return;
                }
                if (!isCompatible(eventLoop)) {
                    promise.setFailure(
                            new IllegalStateException("incompatible event loop type: " + eventLoop.getClass().getName()));
                    return;
                }
    
                AbstractChannel.this.eventLoop = eventLoop;
    
                if (eventLoop.inEventLoop()) {
                    register0(promise);
                } else {
                    try {
                        eventLoop.execute(new Runnable() {
                            @Override
                            public void run() {
                                register0(promise);
                            }
                        });
                    } catch (Throwable t) {
                        logger.warn(
                                "Force-closing a channel whose registration task was not accepted by an event loop: {}",
                                AbstractChannel.this, t);
                        closeForcibly();
                        closeFuture.setClosed();
                        safeSetFailure(promise, t);
                    }
                }
            }

    eventLoop.inEventLoop()判断主线程并不是NioEventLoop唯一绑定的那个线程:

        @Override
        public boolean inEventLoop() {
            return inEventLoop(Thread.currentThread());
        }
    
        @Override
        public boolean inEventLoop(Thread thread) {
            return thread == this.thread;
        }

    此时当前eventLoop将regsiter0放入task队列,通过doStartThread()方法开启新的线程执行:

        @Override
        public void execute(Runnable task) {
            ObjectUtil.checkNotNull(task, "task");
            execute(task, !(task instanceof LazyRunnable) && wakesUpForTask(task));
        }
    
        @Override
        public void lazyExecute(Runnable task) {
            execute(ObjectUtil.checkNotNull(task, "task"), false);
        }
    
        private void execute(Runnable task, boolean immediate) {
            boolean inEventLoop = inEventLoop();
            addTask(task);
            if (!inEventLoop) {
                startThread();
                if (isShutdown()) {
                    boolean reject = false;
                    try {
                        if (removeTask(task)) {
                            reject = true;
                        }
                    } catch (UnsupportedOperationException e) {
                        // The task queue does not support removal so the best thing we can do is to just move on and
                        // hope we will be able to pick-up the task before its completely terminated.
                        // In worst case we will log on termination.
                    }
                    if (reject) {
                        reject();
                    }
                }
            }
    
            if (!addTaskWakesUp && immediate) {
                wakeup(inEventLoop);
            }
        }
    
        private void startThread() {
            if (state == ST_NOT_STARTED) {
                if (STATE_UPDATER.compareAndSet(this, ST_NOT_STARTED, ST_STARTED)) {
                    boolean success = false;
                    try {
                        doStartThread();
                        success = true;
                    } finally {
                        if (!success) {
                            STATE_UPDATER.compareAndSet(this, ST_STARTED, ST_NOT_STARTED);
                        }
                    }
                }
            }
        }
    
        private void doStartThread() {
            assert thread == null;
            // todo 断言线程为空, 然后才创建新的线程
            executor.execute(new Runnable() { // todo 每次Execute 都是在使用 默认的线程工厂,创建一个线程并执行 Runable里面的任务
            @Override
            public void run() {
            // todo 获取刚才创建出来的线程,保存在NioEventLoop中的 thread 变量里面, 这里其实就是在进行那个唯一的绑定
                thread = Thread.currentThread();
                updateLastExecutionTime();
                try {
                    // todo 实际启动线程, 到这里  NioEventLoop 就启动完成了
                    SingleThreadEventExecutor.this.run();
                }
            }
            ......
        }
    

    调用了NioEventLoop的线程执行器的execute,这个方法的源码在下面,可以看到,excute,其实就是在创建线程, 线程创建完成后,立即把新创建出来的线程当作是NioEventLoopbang线程:

    /*
     * Copyright 2013 The Netty Project
     *
     * The Netty Project licenses this file to you under the Apache License,
     * version 2.0 (the "License"); you may not use this file except in compliance
     * with the License. You may obtain a copy of the License at:
     *
     *   http://www.apache.org/licenses/LICENSE-2.0
     *
     * Unless required by applicable law or agreed to in writing, software
     * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
     * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
     * License for the specific language governing permissions and limitations
     * under the License.
     */
    package io.netty.util.concurrent;
    
    import io.netty.util.internal.ObjectUtil;
    
    import java.util.concurrent.Executor;
    import java.util.concurrent.ThreadFactory;
    
    public final class ThreadPerTaskExecutor implements Executor {
        private final ThreadFactory threadFactory;
    
        public ThreadPerTaskExecutor(ThreadFactory threadFactory) {
            this.threadFactory = ObjectUtil.checkNotNull(threadFactory, "threadFactory");
        }
    
        @Override
        public void execute(Runnable command) {
            threadFactory.newThread(command).start();
        }
    }
    

    返回register方法内的register0(),它调用了AbstractNioChannel的doRegister():

        @Override
        protected void doRegister() throws Exception {
            boolean selected = false;
            for (;;) {
                try {
                    selectionKey = javaChannel().register(eventLoop().unwrappedSelector(), 0, this);
                    return;
                } catch (CancelledKeyException e) {
                    if (!selected) {
                        // Force the Selector to select now as the "canceled" SelectionKey may still be
                        // cached and not removed because no Select.select(..) operation was called yet.
                        eventLoop().selectNow();
                        selected = true;
                    } else {
                        // We forced a select operation on the selector before but the SelectionKey is still cached
                        // for whatever reason. JDK bug ?
                        throw e;
                    }
                }
            }
        }

    在这里调用NIO的register()方法完成了真正的注册:

        //java\nio\channels\spi\AbstractSelectableChannel
        public final SelectionKey register(Selector sel, int ops,
                                           Object att)
            throws ClosedChannelException
        {
            synchronized (regLock) {
                if (!isOpen())
                    throw new ClosedChannelException();
                if ((ops & ~validOps()) != 0)
                    throw new IllegalArgumentException();
                if (blocking)
                    throw new IllegalBlockingModeException();
                SelectionKey k = findKey(sel);
                if (k != null) {
                    k.interestOps(ops);
                    k.attach(att);
                }
                if (k == null) {
                    // New registration
                    synchronized (keyLock) {
                        if (!isOpen())
                            throw new ClosedChannelException();
                        k = ((AbstractSelector)sel).register(this, ops, att);
                        addKey(k);
                    }
                }
                return k;
            }
        }

     

  3. SingleThreadEventExecutor.this.run(); 这行代码的意思是,调用本类的Run()方法,这个Run()方法就是真正在干活的事件循环,但是呢, 在本类中,Run()是一个抽象方法,因此我们要去找他的子类,那么是谁重写的这个Run()呢? 就是NioEventLoop, 它根据自己需求,重写了这个方法(删除了部分注解和收尾工作):

        @Override
        protected void run() {
            int selectCnt = 0;
            for (;;) {
                try {
                    int strategy;
                    try {
                        strategy = selectStrategy.calculateStrategy(selectNowSupplier, hasTasks());
                        switch (strategy) {
                        case SelectStrategy.CONTINUE:
                            continue;
    
                        case SelectStrategy.BUSY_WAIT:
                            // fall-through to SELECT since the busy-wait is not supported with NIO
                        //轮询IO事件,获取当前感兴趣的channel
                        case SelectStrategy.SELECT:
                            long curDeadlineNanos = nextScheduledTaskDeadlineNanos();
                            if (curDeadlineNanos == -1L) {
                                curDeadlineNanos = NONE; // nothing on the calendar
                            }
                            nextWakeupNanos.set(curDeadlineNanos);
                            try {
                                if (!hasTasks()) {
                                    strategy = select(curDeadlineNanos);
                                }
                            } finally {
                                // This update is just to help block unnecessary selector wakeups
                                // so use of lazySet is ok (no race condition)
                                nextWakeupNanos.lazySet(AWAKE);
                            }
                            // fall through
                        default:
                        }
                    } catch (IOException e) {
                        // If we receive an IOException here its because the Selector is messed up. Let's rebuild
                        // the selector and retry. https://github.com/netty/netty/issues/8566
                        rebuildSelector0();
                        selectCnt = 0;
                        handleLoopException(e);
                        continue;
                    }
    
                    //处理IO事件
                    selectCnt++;
                    cancelledKeys = 0;
                    needsToSelectAgain = false;
                    final int ioRatio = this.ioRatio;
                    boolean ranTasks;
                    if (ioRatio == 100) {
                        try {
                            if (strategy > 0) {
                                processSelectedKeys();
                            }
                        } finally {
                            // Ensure we always run tasks.
                            ranTasks = runAllTasks();
                        }
                    } else if (strategy > 0) {
                        final long ioStartTime = System.nanoTime();
                        try {
                            processSelectedKeys();
                        } finally {
                            // Ensure we always run tasks.
                            final long ioTime = System.nanoTime() - ioStartTime;
                            ranTasks = runAllTasks(ioTime * (100 - ioRatio) / ioRatio);
                        }
                    } else {
                        ranTasks = runAllTasks(0); // This will run the minimum number of tasks
                    }
    
                    if (ranTasks || strategy > 0) {
                        if (selectCnt > MIN_PREMATURE_SELECTOR_RETURNS && logger.isDebugEnabled()) {
                            logger.debug("Selector.select() returned prematurely {} times in a row for Selector {}.",
                                    selectCnt - 1, selector);
                        }
                        selectCnt = 0;
                    } else if (unexpectedSelectorWakeup(selectCnt)) { // Unexpected wakeup (unusual case)
                        selectCnt = 0;
                    }
                } catch (CancelledKeyException e) {
                    // Harmless exception - log anyway
                    if (logger.isDebugEnabled()) {
                        logger.debug(CancelledKeyException.class.getSimpleName() + " raised by a Selector {} - JDK bug?",
                                selector, e);
                    }
                } catch (Throwable t) {
                    handleLoopException(t);
                }
                // Always handle shutdown even if the loop processing threw an exception.
                try {
                    if (isShuttingDown()) {
                        closeAll();
                        if (confirmShutdown()) {
                            return;
                        }
                    }
                } catch (Throwable t) {
                    handleLoopException(t);
                }
            }
        }

    NioEventLoop.run()做了三件事情:轮询IO事件、处理IO事件、处理非IO任务

  4. strategy = select(curDeadlineNanos);select()称作: 基于deadline的任务穿插处理逻辑

        //NIO Selector
        private Selector selector;
        private int select(long deadlineNanos) throws IOException {
            if (deadlineNanos == NONE) {
                return selector.select();
            }
            // Timeout will only be 0 if deadline is within 5 microsecs
            long timeoutMillis = deadlineToDelayNanos(deadlineNanos + 995000L) / 1000000L;
            return timeoutMillis <= 0 ? selector.selectNow() : selector.select(timeoutMillis);
        }

    如果截止日期在5微秒内,超时将设置成0,视为没有超时;当timeout <= 0代表没有超时,根据是否超时来判断调用NIO selector不同的select()方法来获取感兴趣的channel。

  5. 聚焦IO事件处理代码——processSelectedKeys()方法,感兴趣的事件strategy>0被调用:

        private void processSelectedKeys() {
            if (selectedKeys != null) {
                processSelectedKeysOptimized();
            } else {
                processSelectedKeysPlain(selector.selectedKeys());
            }
        }

    跟进processSelectedKeysOptimized():

        private void processSelectedKeysOptimized() {
            for (int i = 0; i < selectedKeys.size; ++i) {
                final SelectionKey k = selectedKeys.keys[i];
                // null out entry in the array to allow to have it GC'ed once the Channel close
                // See https://github.com/netty/netty/issues/2363
                selectedKeys.keys[i] = null;
    
                final Object a = k.attachment();
    
                if (a instanceof AbstractNioChannel) {
                    processSelectedKey(k, (AbstractNioChannel) a);
                } else {
                    @SuppressWarnings("unchecked")
                    NioTask<SelectableChannel> task = (NioTask<SelectableChannel>) a;
                    processSelectedKey(k, task);
                }
    
                if (needsToSelectAgain) {
                    // null out entries in the array to allow to have it GC'ed once the Channel close
                    // See https://github.com/netty/netty/issues/2363
                    selectedKeys.reset(i + 1);
    
                    selectAgain();
                    i = -1;
                }
            }
        }

    这个selectedKyes是怎么赋值的呢?是在NioEventLoop初始化构造器中:

        NioEventLoop(NioEventLoopGroup parent, Executor executor, SelectorProvider selectorProvider,
                     SelectStrategy strategy, RejectedExecutionHandler rejectedExecutionHandler,
                     EventLoopTaskQueueFactory queueFactory) {
            super(parent, executor, false, newTaskQueue(queueFactory), newTaskQueue(queueFactory),
                    rejectedExecutionHandler);
            this.provider = ObjectUtil.checkNotNull(selectorProvider, "selectorProvider");
            this.selectStrategy = ObjectUtil.checkNotNull(strategy, "selectStrategy");
            final SelectorTuple selectorTuple = openSelector();
            this.selector = selectorTuple.selector;
            this.unwrappedSelector = selectorTuple.unwrappedSelector;
        }

    注意openSelector()方法,openSelector完成了selector的初始化和Set到SelectedSelectionKeySet的包装:

        private SelectorTuple openSelector() {
            final Selector unwrappedSelector;
            try {
                //初始化selector
                unwrappedSelector = provider.openSelector();
            } catch (IOException e) {
                throw new ChannelException("failed to open a new selector", e);
            }
    
            if (DISABLE_KEY_SET_OPTIMIZATION) {
                return new SelectorTuple(unwrappedSelector);
            }
    
            Object maybeSelectorImplClass = AccessController.doPrivileged(new PrivilegedAction<Object>() {
                @Override
                public Object run() {
                    try {
                        return Class.forName(
                                "sun.nio.ch.SelectorImpl",
                                false,
                                PlatformDependent.getSystemClassLoader());
                    } catch (Throwable cause) {
                        return cause;
                    }
                }
            });
    
            if (!(maybeSelectorImplClass instanceof Class) ||
                // ensure the current selector implementation is what we can instrument.
                !((Class<?>) maybeSelectorImplClass).isAssignableFrom(unwrappedSelector.getClass())) {
                if (maybeSelectorImplClass instanceof Throwable) {
                    Throwable t = (Throwable) maybeSelectorImplClass;
                    logger.trace("failed to instrument a special java.util.Set into: {}", unwrappedSelector, t);
                }
                return new SelectorTuple(unwrappedSelector);
            }
    
            final Class<?> selectorImplClass = (Class<?>) maybeSelectorImplClass;
            final SelectedSelectionKeySet selectedKeySet = new SelectedSelectionKeySet();
    
            Object maybeException = AccessController.doPrivileged(new PrivilegedAction<Object>() {
                @Override
                public Object run() {
                    try {
                        Field selectedKeysField = selectorImplClass.getDeclaredField("selectedKeys");
                        Field publicSelectedKeysField = selectorImplClass.getDeclaredField("publicSelectedKeys");
    
                        if (PlatformDependent.javaVersion() >= 9 && PlatformDependent.hasUnsafe()) {
                            // Let us try to use sun.misc.Unsafe to replace the SelectionKeySet.
                            // This allows us to also do this in Java9+ without any extra flags.
                            long selectedKeysFieldOffset = PlatformDependent.objectFieldOffset(selectedKeysField);
                            long publicSelectedKeysFieldOffset =
                                    PlatformDependent.objectFieldOffset(publicSelectedKeysField);
    
                            if (selectedKeysFieldOffset != -1 && publicSelectedKeysFieldOffset != -1) {
                                PlatformDependent.putObject(
                                        unwrappedSelector, selectedKeysFieldOffset, selectedKeySet);
                                PlatformDependent.putObject(
                                        unwrappedSelector, publicSelectedKeysFieldOffset, selectedKeySet);
                                return null;
                            }
                            // We could not retrieve the offset, lets try reflection as last-resort.
                        }
    
                        Throwable cause = ReflectionUtil.trySetAccessible(selectedKeysField, true);
                        if (cause != null) {
                            return cause;
                        }
                        cause = ReflectionUtil.trySetAccessible(publicSelectedKeysField, true);
                        if (cause != null) {
                            return cause;
                        }
    
                        selectedKeysField.set(unwrappedSelector, selectedKeySet);
                        publicSelectedKeysField.set(unwrappedSelector, selectedKeySet);
                        return null;
                    } catch (NoSuchFieldException e) {
                        return e;
                    } catch (IllegalAccessException e) {
                        return e;
                    }
                }
            });
    
            if (maybeException instanceof Exception) {
                selectedKeys = null;
                Exception e = (Exception) maybeException;
                logger.trace("failed to instrument a special java.util.Set into: {}", unwrappedSelector, e);
                return new SelectorTuple(unwrappedSelector);
            }
            selectedKeys = selectedKeySet;
            logger.trace("instrumented a special java.util.Set into: {}", unwrappedSelector);
            return new SelectorTuple(unwrappedSelector,
                                     new SelectedSelectionKeySetSelector(unwrappedSelector, selectedKeySet));
        }

    回到processSelectedKeysOptimized()方法,NioEventLoop是如何在千百条channel中,精确获取出现指定感兴趣事件的channel的?看看这段代码:

    final Object a = k.attachment();

    并且,判断出Key的类型后,执行处理逻辑的代码中的入参都是一样的processSelectedKey(a,k) , 这是在干什么呢?其实,我们知道,每个NioEventLoop开始干活后,会有很多客户端的连接channel前来和它建立连接,一个事件循环同时为多条channel服务,而且一条channel的整个生命周期都只和一个NioEventLoop关联,现在好了,事件循环的选择器轮询出了诸多的channel中有channel出现了感兴趣的事件,下一步处理这个事件的前提得知道,究竟是哪个channel?使用的attachment特性,早在Channel注册进Selector时,进存放进去了,下面是Netty中,Channel注册进Selector的源码

        @Override
        protected void doRegister() throws Exception {
            boolean selected = false;
            for (;;) {
                try {
                    // todo  javaChannel() -- 返回SelectableChanel 可选择的Channel,换句话说,可以和Selector搭配使用,他是channel体系的顶级抽象类, 实际的类型是 ServerSocketChannel
                    // todo  eventLoop().unwrappedSelector(), -- >  获取选择器, 现在在AbstractNioChannel中 获取到的eventLoop是BossGroup里面的
                    // todo  到目前看, 他是把ServerSocketChannel(系统创建的) 注册进了 EventLoop的选择器
         
                    // todo 到目前为止, 虽然注册上了,但是它不关心任何事件
                    selectionKey = javaChannel().register(eventLoop().unwrappedSelector(), 0, this);
                    return;
                } catch (CancelledKeyException e) {
    

    这里的 最后一个参数是 this是当前的channel , 意思是把当前的Channel当成是一个 attachment(附件) 绑定到selector上,之后需要时候再通过selectionkey获取。之后就开启IO事件的处理,也就是对于感兴趣事件的处理:

        private void processSelectedKey(SelectionKey k, AbstractNioChannel ch) {
            final AbstractNioChannel.NioUnsafe unsafe = ch.unsafe();
            if (!k.isValid()) {
                final EventLoop eventLoop;
                try {
                    eventLoop = ch.eventLoop();
                } catch (Throwable ignored) {
                    // If the channel implementation throws an exception because there is no event loop, we ignore this
                    // because we are only trying to determine if ch is registered to this event loop and thus has authority
                    // to close ch.
                    return;
                }
                // Only close ch if ch is still registered to this EventLoop. ch could have deregistered from the event loop
                // and thus the SelectionKey could be cancelled as part of the deregistration process, but the channel is
                // still healthy and should not be closed.
                // See https://github.com/netty/netty/issues/5125
                if (eventLoop == this) {
                    // close the channel if the key is not valid anymore
                    unsafe.close(unsafe.voidPromise());
                }
                return;
            }
    
            try {
                int readyOps = k.readyOps();
                // We first need to call finishConnect() before try to trigger a read(...) or write(...) as otherwise
                // the NIO JDK channel implementation may throw a NotYetConnectedException.
                if ((readyOps & SelectionKey.OP_CONNECT) != 0) {
                    // remove OP_CONNECT as otherwise Selector.select(..) will always return without blocking
                    // See https://github.com/netty/netty/issues/924
                    int ops = k.interestOps();
                    ops &= ~SelectionKey.OP_CONNECT;
                    k.interestOps(ops);
    
                    unsafe.finishConnect();
                }
    
                // Process OP_WRITE first as we may be able to write some queued buffers and so free memory.
                if ((readyOps & SelectionKey.OP_WRITE) != 0) {
                    // Call forceFlush which will also take care of clear the OP_WRITE once there is nothing left to write
                    ch.unsafe().forceFlush();
                }
    
                // Also check for readOps of 0 to workaround possible JDK bug which may otherwise lead
                // to a spin loop
                if ((readyOps & (SelectionKey.OP_READ | SelectionKey.OP_ACCEPT)) != 0 || readyOps == 0) {
                    unsafe.read();
                }
            } catch (CancelledKeyException ignored) {
                unsafe.close(unsafe.voidPromise());
            }
        }

    当以上步骤执行完毕后始终会执行runAllTasks()

  6. runAllTasks之后再分析,有点分析不动了,看源码真的很容易让人放弃QAQ

接收请求

由之前服务器启动过程分析可知:

NioEventLoop.run()中循环处理IO事件,通过void processSelectedKey(SelectionKey k, AbstractNioChannel ch)方法来实际处理,当一个请求进来之后,首先轮询到其IO事件,获取对应的selectionKey,进入unsafe.read():

AbstractNioMessageChannel#NioMessageUnsafe.read

    @Override
        public void read() {
            //判断eventLoop是否是当前线程
            assert eventLoop().inEventLoop();
            final ChannelConfig config = config();
            final ChannelPipeline pipeline = pipeline();
            final RecvByteBufAllocator.Handle allocHandle = unsafe().recvBufAllocHandle();
            allocHandle.reset(config);

            boolean closed = false;
            Throwable exception = null;
            try {
                try {
                    do {
                        int localRead = doReadMessages(readBuf);
                        if (localRead == 0) {
                            break;
                        }
                        if (localRead < 0) {
                            closed = true;
                            break;
                        }

                        allocHandle.incMessagesRead(localRead);
                    } while (allocHandle.continueReading());
                } catch (Throwable t) {
                    exception = t;
                }

                int size = readBuf.size();
                for (int i = 0; i < size; i ++) {
                    readPending = false;
                    pipeline.fireChannelRead(readBuf.get(i));
                }
                readBuf.clear();
                allocHandle.readComplete();
                pipeline.fireChannelReadComplete();

                if (exception != null) {
                    closed = closeOnReadError(exception);

                    pipeline.fireExceptionCaught(exception);
                }

                if (closed) {
                    inputShutdown = true;
                    if (isOpen()) {
                        close(voidPromise());
                    }
                }
            } finally {
                // Check if there is a readPending which was not processed yet.
                // This could be for two reasons:
                // * The user called Channel.read() or ChannelHandlerContext.read() in channelRead(...) method
                // * The user called Channel.read() or ChannelHandlerContext.read() in channelReadComplete(...) method
                //
                // See https://github.com/netty/netty/issues/2254
                if (!readPending && !config.isAutoRead()) {
                    removeReadOp();
                }
            }
        }

进入do while循环执行doReadMessage(),读取bossGourp中NioServerSocketChannel接收到的请求,并把请求放入容器reafBuf

private final List<Object> readBuf = new ArrayList<Object>();

NioServerSocketChannel.doReadMessage()

首先获取通过SocketUtils获取JDK SocketChannel,是对应客户端的channel,用于读写数据,并包装成为Netty的NioSocketChannel后加入readBuf容器

    @Override
    protected int doReadMessages(List<Object> buf) throws Exception {
        SocketChannel ch = SocketUtils.accept(javaChannel());

        try {
            if (ch != null) {
                buf.add(new NioSocketChannel(this, ch));
                return 1;
            }
        } catch (Throwable t) {
            logger.warn("Failed to create a new channel from an accepted socket.", t);

            try {
                ch.close();
            } catch (Throwable t2) {
                logger.warn("Failed to close a socket.", t2);
            }
        }

        return 0;
    }

回到AbstractNioMessageChannel#NioMessageUnsafe.read,开始执行

pipeline.fireChannelRead(readBuf.get(i))

将刚才和客户端建立好的channel传递给pipline,开始执行pipline中ChannelHandlerContext链表中handler。关于ChannelPipline、ChannelHandler、ChannelHandlerContext的详细分析可以查看我的另一篇文章:Netty源码分析:三大核心组件——ChannelPipline、ChannelHandler、ChannelHandlerContext

一直跟进pipeline.fireChannelRead(readBuf.get(i))如下:

     static void invokeChannelRead(final AbstractChannelHandlerContext next, Object msg) {
        final Object m = next.pipeline.touch(ObjectUtil.checkNotNull(msg, "msg"), next);
        EventExecutor executor = next.executor();
        if (executor.inEventLoop()) {
            next.invokeChannelRead(m);
        } else {
            executor.execute(new Runnable() {
                @Override
                public void run() {
                    next.invokeChannelRead(m);
                }
            });
        }
    }

    private void invokeChannelRead(Object msg) {
        if (invokeHandler()) {
            try {
                ((ChannelInboundHandler) handler()).channelRead(this, msg);
            } catch (Throwable t) {
                invokeExceptionCaught(t);
            }
        } else {
            fireChannelRead(msg);
        }
    }

debug如下:

经过多次debug,我们会发现会反复执行invokeChannelRead方法,这是因为目前的pipline中已经有初始化的内置handler处理器了

重点关注一下ServerBootstrapAcceptor

io\netty\bootstrap\ServerBootstrap#ServerBootstrapAcceptor.channelRead:

        @Override
        @SuppressWarnings("unchecked")
        public void channelRead(ChannelHandlerContext ctx, Object msg) {
            final Channel child = (Channel) msg;

            child.pipeline().addLast(childHandler);

            setChannelOptions(child, childOptions, logger);
            setAttributes(child, childAttrs);

            try {
                childGroup.register(child).addListener(new ChannelFutureListener() {
                    @Override
                    public void operationComplete(ChannelFuture future) throws Exception {
                        if (!future.isSuccess()) {
                            forceClose(child, future.cause());
                        }
                    }
                });
            } catch (Throwable t) {
                forceClose(child, t);
            }
        }

这个方法极为重要,干了一件大事:

childGroup.register(child).addListener(new ChannelFutureListener() {
    @Override
    public void operationComplete(ChannelFuture future) throws Exception {
        if (!future.isSuccess()) {
            forceClose(child, future.cause());
        }
    }
});

这个childGroup就是之前初始化的workerGroup,childGroup将和客户端连接好的连接,也就是socketChannel注册了,这就是boosGroup将建立好的连接移交workerGroup的关键代码。

至此,workerGroup循环组就可以开始监听客户端的请求(IO事件),进行业务处理了。

 

参考:

https://blog.csdn.net/qq_31854907/article/details/109331330

 

  • 3
    点赞
  • 4
    收藏
    觉得还不错? 一键收藏
  • 5
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 5
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值