在前面一篇文章中我们介绍了启动Netty的服务端的EventLoopGroup的创建过程以及源码的跟踪，我们了解了它是怎么新建工作线程及入口线程的，接下来我们就看看基础线程建立好以后，我们是如何一步步的启动服务的，我们还是把代码贴一次
public static void main(String[] args) {
EventLoopGroup bossGroup = new NioEventLoopGroup(1);
EventLoopGroup workerGroup = new NioEventLoopGroup();
try {
NettyServerHandlerTest1 handlerTest1 = new NettyServerHandlerTest1();
ServerBootstrap serverBootstrap = new ServerBootstrap();
serverBootstrap.group(bossGroup,workerGroup)
.channel(NioServerSocketChannel.class)
.option(ChannelOption.SO_BACKLOG,1024)
.handler(new LoggingHandler(LogLevel.INFO))
.childHandler(new ChannelInitializer<SocketChannel>() {
@Override
protected void initChannel(SocketChannel socketChannel) throws Exception {
ChannelPipeline pipeline = socketChannel.pipeline();
pipeline.addLast(handlerTest1);
}
});
ChannelFuture channelFuture = serverBootstrap.bind(8010).sync();
channelFuture.channel().closeFuture().sync();
}catch (Exception e) {
e.printStackTrace();
}finally {
bossGroup.shutdownGracefully();
workerGroup.shutdownGracefully();
}
}在上面代码中服务的启动是通过新建一个ServerBootstrap对象作为启动对象的，对象创建完成后我们把bossGroup和workGroup传递给了它，并且使用的是NioServerSocketChannel通道，传递了一个option参数，同时给serverBootstrap添加了一个自定义的handler对象，当然这些只是一些初始化操作，并没有启动服务，真正启动服务的是bind()方法的执行，我们下面就接着看下bind()方法的执行过程
通过serverBootstrap.bind()方法我们经过一些传递方法进入到AbstractBootstrap的doBind(final SocketAddress localAddress)方法中，其实现如下
private ChannelFuture doBind(final SocketAddress localAddress) {
// 初始化channel中的信息
final ChannelFuture regFuture = initAndRegister();
// 获取初始化完成后channel信息
final Channel channel = regFuture.channel();
if (regFuture.cause() != null) {
return regFuture;
}
// 判断是否已经完成初始化
if (regFuture.isDone()) {
// At this point we know that the registration was complete and successful.
ChannelPromise promise = channel.newPromise();
// 使用户自定义的handler能够有效
doBind0(regFuture, channel, localAddress, promise);
return promise;
} else {
// Registration future is almost always fulfilled already, but just in case it's not.
final PendingRegistrationPromise promise = new PendingRegistrationPromise(channel);
regFuture.addListener(new ChannelFutureListener() {
@Override
public void operationComplete(ChannelFuture future) throws Exception {
Throwable cause = future.cause();
if (cause != null) {
// Registration on the EventLoop failed so fail the ChannelPromise directly to not cause an
// IllegalStateException once we try to access the EventLoop of the Channel.
promise.setFailure(cause);
} else {
// Registration was successful, so set the correct executor to use.
// See https://github.com/netty/netty/issues/2586
promise.registered();
doBind0(regFuture, channel, localAddress, promise);
}
}
});
return promise;
}
}
final ChannelFuture initAndRegister() {
Channel channel = null;
try {
// 通过反射的方式创建一个channel
channel = channelFactory.newChannel();
// 初始化channel信息
init(channel);
} catch (Throwable t) {
if (channel != null) {
// channel can be null if newChannel crashed (eg SocketException("too many open files"))
channel.unsafe().closeForcibly();
// as the Channel is not registered yet we need to force the usage of the GlobalEventExecutor
return new DefaultChannelPromise(channel, GlobalEventExecutor.INSTANCE).setFailure(t);
}
// as the Channel is not registered yet we need to force the usage of the GlobalEventExecutor
return new DefaultChannelPromise(new FailedChannel(), GlobalEventExecutor.INSTANCE).setFailure(t);
}
// 通过Java原生的nio进行注册启动初始化工作
ChannelFuture regFuture = config().group().register(channel);
if (regFuture.cause() != null) {
if (channel.isRegistered()) {
channel.close();
} else {
channel.unsafe().closeForcibly();
}
}
// If we are here and the promise is not failed, it's one of the following cases:
// 1) If we attempted registration from the event loop, the registration has been completed at this point.
// i.e. It's safe to attempt bind() or connect() now because the channel has been registered.
// 2) If we attempted registration from the other thread, the registration request has been successfully
// added to the event loop's task queue for later execution.
// i.e. It's safe to attempt bind() or connect() now:
// because bind() or connect() will be executed *after* the scheduled registration task is executed
// because register(), bind(), and connect() are all bound to the same thread.
return regFuture;
}我们看下在initAndRegister()方法中初始化Channel，通过init(channel)方法进去可以看到它是一个抽象方法，它的具体实现分别在ServerBootstrap和Bootstrap中，这里因为看的是ServerBootstrap类，所以我们直接看下它的实现
void init(Channel channel) throws Exception {
// 获取初始化ServerBootstrap时定义的option()方法中的数据
final Map<ChannelOption<?>, Object> options = options0();
synchronized (options) {
setChannelOptions(channel, options, logger);
}
// 获取初始化ServerBootstrap时定义的attr()方法中的数据
final Map<AttributeKey<?>, Object> attrs = attrs0();
synchronized (attrs) {
for (Entry<AttributeKey<?>, Object> e: attrs.entrySet()) {
@SuppressWarnings("unchecked")
AttributeKey<Object> key = (AttributeKey<Object>) e.getKey();
channel.attr(key).set(e.getValue());
}
}
// 获取channel中的一个通道
ChannelPipeline p = channel.pipeline();
final EventLoopGroup currentChildGroup = childGroup;
final ChannelHandler currentChildHandler = childHandler;
final Entry<ChannelOption<?>, Object>[] currentChildOptions;
final Entry<AttributeKey<?>, Object>[] currentChildAttrs;
synchronized (childOptions) {
currentChildOptions = childOptions.entrySet().toArray(newOptionArray(0));
}
synchronized (childAttrs) {
currentChildAttrs = childAttrs.entrySet().toArray(newAttrArray(0));
}
p.addLast(new ChannelInitializer<Channel>() {
@Override
public void initChannel(final Channel ch) throws Exception {
final ChannelPipeline pipeline = ch.pipeline();
ChannelHandler handler = config.handler();
if (handler != null) {
pipeline.addLast(handler);
}
// 添加类似reactor线程模型当中的Acceptor
ch.eventLoop().execute(new Runnable() {
@Override
public void run() {
pipeline.addLast(new ServerBootstrapAcceptor(
ch, currentChildGroup, currentChildHandler, currentChildOptions, currentChildAttrs));
}
});
}
});
}在init()方法中我们主要将一些初始化的信息添加到当前channel中，并对一些公共变量赋值，并且在责任链头部添加了一个类似reactor线程模型的Acceptor对象。初始化完成后回到AbstractBootstrap的initAndRegister()方法的ChannelFuture regFuture = config().group().register(channel)处，我们先看下进入到register方法后的执行顺序图
通过上面的执行图我们可以看到经过一些执行顺序我们直接进入到AbstractChannel类的register(EventLoop eventLoop, final ChannelPromise promise)方法
public final void register(EventLoop eventLoop, final ChannelPromise promise) {
if (eventLoop == null) {
throw new NullPointerException("eventLoop");
}
// 判断是否已经注册过
if (isRegistered()) {
promise.setFailure(new IllegalStateException("registered to an event loop already"));
return;
}
// 判断实例类型
if (!isCompatible(eventLoop)) {
promise.setFailure(
new IllegalStateException("incompatible event loop type: " + eventLoop.getClass().getName()));
return;
}
AbstractChannel.this.eventLoop = eventLoop;
// 判断当前线程是否是已经在运行的线程
if (eventLoop.inEventLoop()) {
// 进行Java nio原生数据封装操作
register0(promise);
} else {
try {
// Executor框架进行Java nio原生数据封装操作
eventLoop.execute(new Runnable() {
@Override
public void run() {
register0(promise);
}
});
} catch (Throwable t) {
logger.warn(
"Force-closing a channel whose registration task was not accepted by an event loop: {}",
AbstractChannel.this, t);
closeForcibly();
closeFuture.setClosed();
safeSetFailure(promise, t);
}
}
}在上面的register()注册代码中，先是判断是否已经初始化过eventloop对象，在检查当前的channel是否已经注册过，防止重复注册，最后判断eventLoop的启动线程是否是当前线程，是的话就直接进行Java nio的元素数据封装操作，否则通过Executor线程框架进行Java nio的原生数据封装操作，当代码启动时通过代码跟踪它是直接执行线程的方式启动的，我们先从这个主线走下去，稍后回来看register0()方法。通过代码跟踪我们在线程框架的情况下直接进入了SingleThreadEventExecutor.execute(Runnable task)方法
public void execute(Runnable task) {
if (task == null) {
throw new NullPointerException("task");
}
// 判断当前线程是否是已经初始化执行的线程
boolean inEventLoop = inEventLoop();
// 将当前任务添加到taskQueue中，taskQueue的初始化是在定义EventLoopGroup的时候定义的
addTask(task);
if (!inEventLoop) {
// 开始执行线程
startThread();
// 如果已经执行关闭操作
if (isShutdown()) {
boolean reject = false;
try {
// 从taskQueue中移除task
if (removeTask(task)) {
reject = true;
}
} catch (UnsupportedOperationException e) {
// The task queue does not support removal so the best thing we can do is to just move on and
// hope we will be able to pick-up the task before its completely terminated.
// In worst case we will log on termination.
}
if (reject) {
reject();
}
}
}
// 往taskQueue队列中添加一个空的线程，默认情况下addTaskWarksUp是false，也是在定义
// EventLoopGroup的时候初始化的
if (!addTaskWakesUp && wakesUpForTask(task)) {
wakeup(inEventLoop);
}
}private void startThread() {
// 判断当前线程是否已经启动过
if (state == ST_NOT_STARTED) {
// 通过CAS的方式再次确认
if (STATE_UPDATER.compareAndSet(this, ST_NOT_STARTED, ST_STARTED)) {
try {
// 执行线程
doStartThread();
} catch (Throwable cause) {
STATE_UPDATER.set(this, ST_NOT_STARTED);
PlatformDependent.throwException(cause);
}
}
}
}private void doStartThread() {
assert thread == null;
executor.execute(new Runnable() {
@Override
public void run() {
thread = Thread.currentThread();
if (interrupted) {
thread.interrupt();
}
boolean success = false;
// 更新执行时间
updateLastExecutionTime();
try {
// 执行添加的线程
SingleThreadEventExecutor.this.run();
success = true;
} catch (Throwable t) {
logger.warn("Unexpected exception from an event executor: ", t);
} finally {
for (;;) {
int oldState = state;
// 如果关闭了则跳出循环
if (oldState >= ST_SHUTTING_DOWN || STATE_UPDATER.compareAndSet(
SingleThreadEventExecutor.this, oldState, ST_SHUTTING_DOWN)) {
break;
}
}
// Check if confirmShutdown() was called at the end of the loop.
if (success && gracefulShutdownStartTime == 0) {
if (logger.isErrorEnabled()) {
logger.error("Buggy " + EventExecutor.class.getSimpleName() + " implementation; " +
SingleThreadEventExecutor.class.getSimpleName() + ".confirmShutdown() must " +
"be called before run() implementation terminates.");
}
}
try {
// Run all remaining tasks and shutdown hooks.
for (;;) {
// 确认关闭则跳出循环
if (confirmShutdown()) {
break;
}
}
} finally {
try {
// 关闭通道选择器或者一些清理操作
cleanup();
} finally {
// Lets remove all FastThreadLocals for the Thread as we are about to terminate and notify
// the future. The user may block on the future and once it unblocks the JVM may terminate
// and start unloading classes.
// See https://github.com/netty/netty/issues/6596.
FastThreadLocal.removeAll();
STATE_UPDATER.set(SingleThreadEventExecutor.this, ST_TERMINATED);
// 锁释放
threadLock.release();
if (!taskQueue.isEmpty()) {
if (logger.isWarnEnabled()) {
logger.warn("An event executor terminated with " +
"non-empty task queue (" + taskQueue.size() + ')');
}
}
terminationFuture.setSuccess(null);
}
}
}
}
});
}通过SingleThreadEventExecutor的三段代码就基本上完成了任务的启动操作的判断，如果要启动doStartThread()方法还得经ThreadPerTaskExecutor类实现的executor()方法，启动了doStartThread()的线程后，通过SingleThreadEventExecutor.this.run()方法的具体实现类NioEventLoop的run()方法
protected void run() {
for (;;) {
try {
try {
// 对当前的taskQueue和tailTasks中执行的线程状态进行判断
switch (selectStrategy.calculateStrategy(selectNowSupplier, hasTasks())) {
case SelectStrategy.CONTINUE:
continue;
case SelectStrategy.BUSY_WAIT:
// fall-through to SELECT since the busy-wait is not supported with NIO
case SelectStrategy.SELECT:
// 进行Java nio 原生的select开启
select(wakenUp.getAndSet(false));
// 'wakenUp.compareAndSet(false, true)' is always evaluated
// before calling 'selector.wakeup()' to reduce the wake-up
// overhead. (Selector.wakeup() is an expensive operation.)
//
// However, there is a race condition in this approach.
// The race condition is triggered when 'wakenUp' is set to
// true too early.
//
// 'wakenUp' is set to true too early if:
// 1) Selector is waken up between 'wakenUp.set(false)' and
// 'selector.select(...)'. (BAD)
// 2) Selector is waken up between 'selector.select(...)' and
// 'if (wakenUp.get()) { ... }'. (OK)
//
// In the first case, 'wakenUp' is set to true and the
// following 'selector.select(...)' will wake up immediately.
// Until 'wakenUp' is set to false again in the next round,
// 'wakenUp.compareAndSet(false, true)' will fail, and therefore
// any attempt to wake up the Selector will fail, too, causing
// the following 'selector.select(...)' call to block
// unnecessarily.
//
// To fix this problem, we wake up the selector again if wakenUp
// is true immediately after selector.select(...).
// It is inefficient in that it wakes up the selector for both
// the first case (BAD - wake-up required) and the second case
// (OK - no wake-up required).
if (wakenUp.get()) {
selector.wakeup();
}
// fall through
default:
}
} catch (IOException e) {
// If we receive an IOException here its because the Selector is messed up. Let's rebuild
// the selector and retry. https://github.com/netty/netty/issues/8566
// 产生异常时重新尝试selector的开启
rebuildSelector0();
handleLoopException(e);
continue;
}
cancelledKeys = 0;
needsToSelectAgain = false;
final int ioRatio = this.ioRatio;
// io执行比率为100
if (ioRatio == 100) {
try {
// 进行selectKey事件的对应处理
processSelectedKeys();
} finally {
// Ensure we always run tasks.
// 运行taskQueue和tailTasks中所有的线程
runAllTasks();
}
} else {
final long ioStartTime = System.nanoTime();
try {
processSelectedKeys();
} finally {
// Ensure we always run tasks.
final long ioTime = System.nanoTime() - ioStartTime;
// 有过期时间限制的执行taskQueue和tailTasks中所有的线程
runAllTasks(ioTime * (100 - ioRatio) / ioRatio);
}
}
} catch (Throwable t) {
handleLoopException(t);
}
// Always handle shutdown even if the loop processing threw an exception.
try {
if (isShuttingDown()) {
closeAll();
if (confirmShutdown()) {
return;
}
}
} catch (Throwable t) {
handleLoopException(t);
}
}
}在run()方法中它是一直进行for无限循环的，这样它就可以不断去获取taskQueue或者tailQueue队列中的线程及线程执行情况，在for循环中首先对当前的taskQueue和tailTask中执行的线程状态进行判断，如果是处于select状态的话，通过方法select(wakenUp.getAndSet(false))进行Java nio 原生的select进行初始化并开启，线程状态判断完成后，判断ioRatio的io比率是否为100，如果是，则通过processSelectedKeys()方法直接进行selectKey事件的处理，并且在finally中经过runAllTasks()方法运行taskQueue和tailTasks中的所有线程，如果不满足比率为100时则同样根据processSelectedKeys()方法获取调用selectKey事件处理的时间进行有时间限制的runAllTasks()方法运行taskQueue和tailTasks中的所有线程，我们先看下processSelectedKeys()方法的实现
private void processSelectedKeys() {
// 判断selectedKeys set集合是否为空，该值可以在NioEventLoop构造函数调用openSelector()方
// 法或run()方法异常时通过rebuildSelector0()中调用openSelector()进行赋值
if (selectedKeys != null) {
// 直接进行Java nio原生操作,如果没有正确打开selector，则重新打开获取selectKey
processSelectedKeysOptimized();
} else {
// selector的值也是在NioEventLoop构造函数中通过openSelector()时进行定义的
// 在直接进行Java nio原生操作上，在selectKey有多个值且如果没有正确打开selector，则重
// 新打开获取selectKey
processSelectedKeysPlain(selector.selectedKeys());
}
}private void processSelectedKeysOptimized() {
for (int i = 0; i < selectedKeys.size; ++i) {
final SelectionKey k = selectedKeys.keys[i];
// null out entry in the array to allow to have it GC'ed once the Channel close
// See https://github.com/netty/netty/issues/2363
selectedKeys.keys[i] = null;
final Object a = k.attachment();
if (a instanceof AbstractNioChannel) {
// 进行具体的Java nio原生SelectionKey判断操作
processSelectedKey(k, (AbstractNioChannel) a);
} else {
@SuppressWarnings("unchecked")
NioTask<SelectableChannel> task = (NioTask<SelectableChannel>) a;
// 进行具体的Java nio原生的NioTask及SelectionKey判断操作
processSelectedKey(k, task);
}
// 是否需要重新打开selector
if (needsToSelectAgain) {
// null out entries in the array to allow to have it GC'ed once the Channel close
// See https://github.com/netty/netty/issues/2363
selectedKeys.reset(i + 1);
selectAgain();
i = -1;
}
}
}private void processSelectedKeysPlain(Set<SelectionKey> selectedKeys) {
// check if the set is empty and if so just return to not create garbage by
// creating a new Iterator every time even if there is nothing to process.
// See https://github.com/netty/netty/issues/597
if (selectedKeys.isEmpty()) {
return;
}
Iterator<SelectionKey> i = selectedKeys.iterator();
for (;;) {
final SelectionKey k = i.next();
final Object a = k.attachment();
// 消除当前的selectionKey
i.remove();
if (a instanceof AbstractNioChannel) {
processSelectedKey(k, (AbstractNioChannel) a);
} else {
@SuppressWarnings("unchecked")
NioTask<SelectableChannel> task = (NioTask<SelectableChannel>) a;
processSelectedKey(k, task);
}
// 是否还有值
if (!i.hasNext()) {
break;
}
if (needsToSelectAgain) {
selectAgain();
selectedKeys = selector.selectedKeys();
// Create the iterator again to avoid ConcurrentModificationException
if (selectedKeys.isEmpty()) {
break;
} else {
i = selectedKeys.iterator();
}
}
}
}在processSelectedKeys()方法及相关方法中进行了SelectionKey的事件处理操作，操作底层都是使用Java nio的原生类进行操作的，这里就不在贴出来了，我们接着看下runAllTasks(ioTime * (100 - ioRatio) / ioRatio)方法
protected boolean runAllTasks(long timeoutNanos) {
// 获取定时任务执行的线程到taskQueue中
fetchFromScheduledTaskQueue();
// 获取taskQueue中线程
Runnable task = pollTask();
if (task == null) {
// 执行所有在tailTasks中的线程
afterRunningAllTasks();
return false;
}
final long deadline = ScheduledFutureTask.nanoTime() + timeoutNanos;
long runTasks = 0;
long lastExecutionTime;
for (;;) {
// 启动线程
safeExecute(task);
runTasks ++;
// Check timeout every 64 tasks because nanoTime() is relatively expensive.
// XXX: Hard-coded value - will make it configurable if it is really a problem.
if ((runTasks & 0x3F) == 0) {
lastExecutionTime = ScheduledFutureTask.nanoTime();
// 判断是否已经超过指定的定长时间
if (lastExecutionTime >= deadline) {
break;
}
}
// 从taskQueue中移除线程
task = pollTask();
if (task == null) {
lastExecutionTime = ScheduledFutureTask.nanoTime();
break;
}
}
// 执行所有在tailTasks中的线程
afterRunningAllTasks();
this.lastExecutionTime = lastExecutionTime;
return true;
}通过上面的线程执行就完成了整个流程的服务端的启动，我们这里还要回到AbstractChannel.AbstractUnsafe.register0()方法中看看是如何注册的
private void register0(ChannelPromise promise) {
try {
// check if the channel is still open as it could be closed in the mean time when the register
// call was outside of the eventLoop
if (!promise.setUncancellable() || !ensureOpen(promise)) {
return;
}
boolean firstRegistration = neverRegistered;
// 调用Java nio原生register进行注册操作
doRegister();
neverRegistered = false;
registered = true;
// Ensure we call handlerAdded(...) before we actually notify the promise. This is needed as the
// user may already fire events through the pipeline in the ChannelFutureListener.
pipeline.invokeHandlerAddedIfNeeded();
safeSetSuccess(promise);
// 对ChannelInboundHandler.channelRegistered()的封装操作
pipeline.fireChannelRegistered();
// Only fire a channelActive if the channel has never been registered. This prevents firing
// multiple channel actives if the channel is deregistered and re-registered.
if (isActive()) {
// 判断是否已经注册过
if (firstRegistration) {
// 对ChannelInboundHandler.channelRegistered()的封装操作
pipeline.fireChannelActive();
} else if (config().isAutoRead()) {// 是否启用了自动读取操作
// This channel was registered before and autoRead() is set. This means we need to begin read
// again so that we process inbound data.
//
// See https://github.com/netty/netty/issues/4805
// 如果在没有已经绑定的情况下，也就是初始化的时候，那么selectionKey初
// 始化为SelectionKey.OP_READ
beginRead();
}
}
} catch (Throwable t) {
// Close the channel directly to avoid FD leak.
closeForcibly();
closeFuture.setClosed();
safeSetFailure(promise, t);
}
}在register0()方法中我们完成了Java nio原生方法的注册调用以及ChannelInboundHandler的注册工作，同时对注册和没有注册情况时的不同操作判断，以上就是所有的启动过程。需要注意的是如果跟踪代码的时候断点太多可能不会进入到register0()方法，可以少用点断点直接进入到register0()方法观察。
533
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
