Netty启动分析

NIO SelectorImpl分析

java Channel注册到Selector过程时候，会调用AbstractSelectableChannel的register方法,第三个参数是附件attachment，即this=NioServerSocketChannel和NioSocketChannel的父类AbstractNioChannel

selectionKey = javaChannel().register(eventLoop().unwrappedSelector(), 0, this);

然后AbstractSelectableChannel调用SelectorImpl的register方法,将服务端NioServerSocketChannel或者客户端NioSocketChannel添加到SelectionKey相关附件attachment中，这样有了key就能获取其SocketChannel

public final SelectionKey register(Selector sel, int ops,
                                   Object att)

Netty前置知识

简单总结

新建两个线程组，boss线程组启动一条线程监听OP_ACCEPT事件，worker线程组默认启动cpu核数*2的线程,监听客户端连接的OP_READ和OP_WRITE事件,处理IO事件，如何传递的channel呢？

• boosGroup：

使用NioEventLoop启动run()，循环坚挺selector事件，异步处理包括：一、注册channel到selector里边，接收到key后，二、获取channel，pinple找到ServerBootstrap，进行workGroup NioEventLoop异步处理。

• workGroup

NioEventLoop启动run()，循环坚挺selector事件，进行read处理，pipelile找到的是我们注册的handler进行处理

知识点

• 每一个NioEventLoop都有一个Selector,openSelector()反射sun.nio.ch.SelectorImpl修改成数组SelectionKey[] keys

• 每一个Channel都会有一个DefaultChannelPipeline

• DefaultChannelPipeline添加handler到pipeline双向链表中，服务端启动过程添加ServerBootstrapAcceptor作为handler

• Channel需要注册到NioEventLoop的Selector里边

• NioEventLoop父类SingleThreadEventExecutor添加offer任务队列Queue taskQueue

  • 启动时候先添加register0注册channel到selector任务
  • 之后pipeline调用通过AbstractChannelHandlerContext ctx获取channel调用添加ServerBootstrapAcceptor到pipeline的task任务

• 任务通过NioEventLoop 的run方法进行调用safeExecute从队列获取进行异步启动线程执行的

服务端分析NioEventLoop

服务端代码

package com.example.gateway;

import io.netty.bootstrap.ServerBootstrap;
import io.netty.buffer.ByteBuf;
import io.netty.buffer.Unpooled;
import io.netty.channel.*;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioServerSocketChannel;
import io.netty.handler.codec.http.*;
import io.netty.util.CharsetUtil;

public class NettyServer {
    public static void main(String[] args) throws Exception {
        EventLoopGroup bossGroup = new NioEventLoopGroup();
        EventLoopGroup workerGroup = new NioEventLoopGroup();
        try {

            ServerBootstrap serverBootstrap = new ServerBootstrap();
            serverBootstrap.group(bossGroup, workerGroup).channel(NioServerSocketChannel.class)
                    .childHandler(new TestServerInitializer());
            ChannelFuture channelFuture = serverBootstrap.bind(8888).sync();
            channelFuture.channel().closeFuture().sync();

        } finally {
            bossGroup.shutdownGracefully();
            workerGroup.shutdownGracefully();

        }

    }
}

class TestServerInitializer extends ChannelInitializer<SocketChannel> {
    @Override
    protected void initChannel(SocketChannel ch) throws Exception {
        ChannelPipeline pipeline = ch.pipeline();
        pipeline.addLast("httpServerCodec", new HttpServerCodec());
        pipeline.addLast("testHttpServerHandler", new TestHttpServerHandler());
    }
}

class TestHttpServerHandler extends SimpleChannelInboundHandler<HttpObject> {
    @Override
    protected void channelRead0(ChannelHandlerContext ctx, HttpObject msg) throws Exception {
        if (msg instanceof HttpRequest) {
            ByteBuf content = Unpooled.copiedBuffer("Hello, World", CharsetUtil.UTF_8);
            FullHttpResponse response = new DefaultFullHttpResponse(HttpVersion.HTTP_1_1, HttpResponseStatus.OK, content);
            response.headers().set(HttpHeaderNames.CONTENT_TYPE, "text/plain");
            response.headers().set(HttpHeaderNames.CONTENT_LENGTH, content.readableBytes());
            ctx.writeAndFlush(response);
        }

    }
}

继承关系

NioEventLoop extends SingleThreadEventLoop

执行next获取NioEventLoop执行时序图：

bind（） - AbstractBootstrap initAndRegister（） - MultithreadEventLoopGroup register（） - SingleThreadEventLoop register（） - AbstractChannel register（）

SingleThreadEventLoop register

@Override
public ChannelFuture register(final ChannelPromise promise) {
    ObjectUtil.checkNotNull(promise, "promise");
    promise.channel().unsafe().register(this, promise);
    return promise;
}

AbstractChannel register

eventLoop.execute(new Runnable() {
    @Override
    public void run() {
        register0(promise);
    }
});

NioEventLoop本身是一个时间循环器ExecutorService，会提交执行execute线程

因为NioEventLoop继承了SingleThreadEventLoop类，SingleThreadEventLoop继承SingleThreadEventExecutor类，SingleThreadEventExecutor实现了Executor接口，所以调用execute时候会调用SingleThreadEventExecutor的execute方法。

@Override
public void execute(Runnable task) {
    if (task == null) {
        throw new NullPointerException("task");
    }

    boolean inEventLoop = inEventLoop();
    if (inEventLoop) {
        addTask(task);
    } else {
        startThread();
        addTask(task);
        if (isShutdown() && removeTask(task)) {
            reject();
        }
    }

    if (!addTaskWakesUp && wakesUpForTask(task)) {
        wakeup(inEventLoop);
    }
}

第一步startThread

使用MultithreadEventLoopGroup创建EventLoop时候传入的ThreadPerTaskExecutor(DefaultThreadFactory factory)线程执行工厂进行创建启动线程,最后执行SingleThreadEventExecutor.this.run();，也就是当前NioEventLoop run()方法，进行IO操作

private void doStartThread() {
    assert thread == null;
    executor.execute(new Runnable() {
        @Override
        public void run() {
            ....
            try {
                SingleThreadEventExecutor.this.run();
                success = true;
            } catch (Throwable t) {
                logger.warn("Unexpected exception from an event executor: ", t);
            } finally {
             ....
        }
    });
}

Run（）方法，刚开始进行selectStrategy.calculateStrategy底层是selectNow调用（刚开始立即返回）和task检查（刚开始是检查channel注册的任务即register0(promise)，通过执行runAllTasks（）调用register0任务

@Override
protected void run() {
    for (;;) {
        try {
            switch (selectStrategy.calculateStrategy(selectNowSupplier, hasTasks())) {
                case SelectStrategy.CONTINUE:
                    continue;
                case SelectStrategy.SELECT:
                    select(wakenUp.getAndSet(false));
                    if (wakenUp.get()) {
                        selector.wakeup();
                    }
                    // fall through
                default:
            }

            cancelledKeys = 0;
            needsToSelectAgain = false;
            final int ioRatio = this.ioRatio;
            if (ioRatio == 100) {
                try {
                    processSelectedKeys();
                } finally {
                    // Ensure we always run tasks.
                    runAllTasks();
                }
            } else {
                final long ioStartTime = System.nanoTime();
                try {
                    processSelectedKeys();
                } finally {
                    // Ensure we always run tasks.
                    final long ioTime = System.nanoTime() - ioStartTime;
                    runAllTasks(ioTime * (100 - ioRatio) / ioRatio);
                }
            }
        } catch (Throwable t) {
            handleLoopException(t);
        }
        // Always handle shutdown even if the loop processing threw an exception.
        try {
            if (isShuttingDown()) {
                closeAll();
                if (confirmShutdown()) {
                    return;
                }
            }
        } catch (Throwable t) {
            handleLoopException(t);
        }
    }
}

之后执行AbstractNioChannel doRegister()，里边包含两步

第一步：将channel注册到NioEventLoop的selectKey上

selectionKey = javaChannel().register(eventLoop().unwrappedSelector(), 0, this);

第二步：接着管道出发添加handler到pipeline中，服务端添加ServerBootstrapAcceptor

pipeline.invokeHandlerAddedIfNeeded();

第二步addTask

SingleThreadEventExecutor有任务队列

protected Queue<Runnable> newTaskQueue(int maxPendingTasks) {
    return new LinkedBlockingQueue<Runnable>(maxPendingTasks);
}

添加任务

• 第一次添加的任务是channel注册任务，通过NioEventLoop父类SingleThreadeventExecutor调用runAllTasks进行执行
• 第二次添加ServerBootstrapAcceptor到pipeline中，通过NioEventLoop父类SingleThreadeventExecutor调用runAllTasks进行执行

客户端发送数据，服务端处理

NioEventLoop的run方法调用processSelectedKeysOptimized找到可用selectKeys轮训，找到key之后调用NioEventLoop processSelectedKeys（key）方法进行处理，

if ((readyOps & (SelectionKey.OP_READ | SelectionKey.OP_ACCEPT)) != 0 || readyOps == 0) {
    unsafe.read();
}

调用NioSocketChannel父类AbstractNioByteChannel的read（）方法，read里边调用doReadMessages方法（），new NioSocketChannel（）放入readBuf里边

private final List<Object> readBuf = new ArrayList<Object>();

protected int doReadMessages(List<Object> buf) throws Exception {
    SocketChannel ch = SocketUtils.accept(javaChannel());

    try {
        if (ch != null) {
            buf.add(new NioSocketChannel(this, ch));
            return 1;
        }
    } catch (Throwable t) {
        logger.warn("Failed to create a new channel from an accepted socket.", t);

        try {
            ch.close();
        } catch (Throwable t2) {
            logger.warn("Failed to close a socket.", t2);
        }
    }

    return 0;
}

之后通过readBuf大小遍历处理

int size = readBuf.size();
for (int i = 0; i < size; i ++) {
    readPending = false;
    pipeline.fireChannelRead(readBuf.get(i));
}

进行管道处理DefaultChannelPipeline fireChannelRead（msg）

@Override
public final ChannelPipeline fireChannelRead(Object msg) {
    AbstractChannelHandlerContext.invokeChannelRead(head, msg);
    return this;
}

AbstractChannelHandlerContext executor获取channel().envenLoop()，即EventExecutor的子类NioEventLoop，然后调用next.invokeChannelRead(m);方法

static void invokeChannelRead(final AbstractChannelHandlerContext next, Object msg) {
    final Object m = next.pipeline.touch(ObjectUtil.checkNotNull(msg, "msg"), next);
    EventExecutor executor = next.executor();
    if (executor.inEventLoop()) {
        next.invokeChannelRead(m);
    } else {
        executor.execute(new Runnable() {
            @Override
            public void run() {
                next.invokeChannelRead(m);
            }
        });
    }
}

invokeChannelRead进行pipeline双向链表执行，找到handler进行处理，执行DefaultChannelPipeline的channelRead方法

@Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
    ctx.fireChannelRead(msg);
}

然后找到下一个handler ServerBootstrap进行处理,childGroup子线程组，加入child即传入过来的new NioSocketChannel(this, ch)

childGroup.register(child).addListener(new ChannelFutureListener() {
    @Override
    public void operationComplete(ChannelFuture future) throws Exception {
        if (!future.isSuccess()) {
            forceClose(child, future.cause());
        }
    }
});

然后重复线程组添加任务操作

• 将BossGroup传入的NioSocketChannel 进行addTask()添加队列

• channel注册到selector

• 管道处理next（）

客户端分析

addTask不同

因为启动时后实例化的是Bootstrap类，所以第二次添加taskQueue是doConnect（）

private static void doConnect(
        final SocketAddress remoteAddress, final SocketAddress localAddress, final ChannelPromise connectPromise) {

    // This method is invoked before channelRegistered() is triggered.  Give user handlers a chance to set up
    // the pipeline in its channelRegistered() implementation.
    final Channel channel = connectPromise.channel();
    channel.eventLoop().execute(new Runnable() {
        @Override
        public void run() {
            if (localAddress == null) {
                channel.connect(remoteAddress, connectPromise);
            } else {
                channel.connect(remoteAddress, localAddress, connectPromise);
            }
            connectPromise.addListener(ChannelFutureListener.CLOSE_ON_FAILURE);
        }
    });
}

Tail 管道连接

AbstractChannelHandlerContext connect（）建立连接

DefaultChannelPipeline connect()，unsafe是AbstractNioUnsafe类

@Override
public void connect(
        ChannelHandlerContext ctx,
        SocketAddress remoteAddress, SocketAddress localAddress,
        ChannelPromise promise) throws Exception {
    unsafe.connect(remoteAddress, localAddress, promise);
}