前言
都说要学一门技术，要先从基础学起，先学会怎么用，再去研究为什么这么用，再去理解源码，今天我反其道而行之一下，从源码开始，学习一门技术：Netty
众所周知，Netty是一个高性能的网络编程框架，Netty提供异步的、事件驱动的网络应用程序框架和工具，用以快速开发高性能、高可靠性的网络服务器和客户端程序。我们常用的框架包括spring，rocketmq，dubbo，elasticsearch等等，这些框架的网络通信部分或多或少都使用到了Netty，那么Netty为什么那么好用，我们从源码开始学起。
在开始学习源码之前，我们有必要知道，Netty是构建于Java的NIO基础之上的，Netty的许多组件和类和NIO都有着相似的地方和相似的作用，首先解释一些Netty中比较重要的组件和类，以免一下子看源码的时候不知道这是用来做什么的。
- Channel：Netty的Channel和NIO的Channel基本相似，Channel代表的一个连接，每个client请求会对应到具体的一个Channel。
-
ChannelFuture：ChannelFuture是Channel异步IO操作的结果。Netty中的所有IO操作都是异步的。这意味着任何IO调用都将立即返回，而不能保证所请求的IO操作在调用结束时完成。相反，将返回一个带有ChannelFuture的实例，该实例将提供有关IO操作的结果或状态的信息。
-
ChannelHandler：Netty定义了良好的类型层次结构来表示不同的处理程序类型，所有的类型的父类是ChannelHandler。ChannelHandler提供了在其生命周期内添加或从ChannelPipeline中删除的方法。
-
ChannelPipeline：ChannelPipeline是一个拦截流经Channel的入站和出站事件的Channel-Handler实例链。
-
Selector：和NIO里的Selector的作用一样，作为一个多路复用器存在。
-
BootStrap：BootStrap是Netty的启动器，是 Netty 提供的一个便利的工厂类，可以通过它来完成 Netty 的客户端或服务器端的 Netty 初始化。BootStrap又分为ServerBootStrap和BootStrap两种，分别对应服务端的启动器和客户端的启动器。
-
EventLoopGroup：Netty是基于事件的模型，所以，EventLoopGroup，EventLoopGroup负责管理Channel的事件处理任务，继承自java.util.concurrent包下的Executor，所以其结构类似与线程池，管理多个EventLoop。
-
EventLoop：EventLoop将由一个永远不会改变的Thread驱动，同时任务（Runnable或者Callable）可以直接提交给EventLoop实现。另外，一个EventLoop可能会被指派用于服务多个Channel，Channel对应的I/O事件（或者说它绑定的ChannelHandler处理器）都由该EventLoop的Thread进行处理。
开始
开始阅读源码前的准备工作当然是下载源码并编译，这个步骤网上都有，我就省去了。我用的源码版本是github上的4.1分支。
源码的工程结构如下
可以看到这里面有好多的模块，由于我也是刚开始学netty，而且里面的很多模块一般也用不到，所以我们就看主要的模块，我们看到netty-example模块，这个模块里都是Netty官方写的一些例子，相信这也是Netty给我这样的新手学习的时候用的，我们也从这个模块入手，我们找到这个模块下的echo包，先来探究一下Netty服务是怎么启动的，我们来看包下的EchoServer这个类。
为了了解ServerBootstrap的启动流程，我们打下两个断点，分别在52行和74行
bossGroup可以看做是一个主线程池，用来接收client请求，workerGroup则可以看做工作线程池，用来处理具体的读写操作。
当我们把断点打在52行之后我们一路点进去，会来到MultithreadEventExecutorGroup这个类，看名字，是多线程的事件处理器组，我们来看下它的构造方法
接下来主要看newChild方法，这是个抽象方法，因为是在构造NIOEventLoopGroup，所以我们进到NIOEventLoopGroup的实现
@Override
protected EventLoop newChild(Executor executor, Object... args) throws Exception {
EventLoopTaskQueueFactory queueFactory = args.length == 4 ? (EventLoopTaskQueueFactory) args[3] : null;
return new NioEventLoop(this, executor, (SelectorProvider) args[0],
((SelectStrategyFactory) args[1]).newSelectStrategy(), (RejectedExecutionHandler) args[2], queueFactory);
}NioEventLoop(NioEventLoopGroup parent, Executor executor, SelectorProvider selectorProvider,
SelectStrategy strategy, RejectedExecutionHandler rejectedExecutionHandler,
EventLoopTaskQueueFactory queueFactory) {
super(parent, executor, false, newTaskQueue(queueFactory), newTaskQueue(queueFactory),
rejectedExecutionHandler);
this.provider = ObjectUtil.checkNotNull(selectorProvider, "selectorProvider");
this.selectStrategy = ObjectUtil.checkNotNull(strategy, "selectStrategy");
//打开Selctor，这行重要
final SelectorTuple selectorTuple = openSelector();
this.selector = selectorTuple.selector;
this.unwrappedSelector = selectorTuple.unwrappedSelector;
}//重要的代码
final Selector unwrappedSelector;
try {
unwrappedSelector = provider.openSelector();
} catch (IOException e) {
throw new ChannelException("failed to open a new selector", e);
}看到这里我们知道，在创建NIOEventLoopGroup的时候会选择并打开一个selector。
接着来看后面的代码，当看到74行
// Start the server.
ChannelFuture f = b.bind(PORT).sync();代表服务器从这里启动，我们看到这里先是绑定了端口，然后调用sync异步等待启动完成，我们先进bind方法，然后来到了io.netty.bootstrap.AbstractBootstrap#doBind方法
//初始化channel然后注册到EventLoopGroup
final ChannelFuture regFuture = initAndRegister();
final Channel channel = regFuture.channel();
if (regFuture.cause() != null) {
return regFuture;
}
//因为是异步执行，所以到这里不一定已经完成
if (regFuture.isDone()) {
// At this point we know that the registration was complete and successful.
ChannelPromise promise = channel.newPromise();
doBind0(regFuture, channel, localAddress, promise);
return promise;
} else {
// Registration future is almost always fulfilled already, but just in case it's not.
final PendingRegistrationPromise promise = new PendingRegistrationPromise(channel);
//在这里等register执行完成来通知再执行bind，这里添加了一个监听器来监听是否完成
regFuture.addListener(new ChannelFutureListener() {
@Override
public void operationComplete(ChannelFuture future) throws Exception {
Throwable cause = future.cause();
if (cause != null) {
// Registration on the EventLoop failed so fail the ChannelPromise directly to not cause an
// IllegalStateException once we try to access the EventLoop of the Channel.
promise.setFailure(cause);
} else {
// Registration was successful, so set the correct executor to use.
// See https://github.com/netty/netty/issues/2586
promise.registered();
doBind0(regFuture, channel, localAddress, promise);
}
}
});
return promise;
}先来看initAndRegister()方法
Channel channel = null;
try {
//用工厂创建了一个channel
channel = channelFactory.newChannel();
//初始化channel
init(channel);
} catch (Throwable t) {
if (channel != null) {
// channel can be null if newChannel crashed (eg SocketException("too many open files"))
channel.unsafe().closeForcibly();
// as the Channel is not registered yet we need to force the usage of the GlobalEventExecutor
return new DefaultChannelPromise(channel, GlobalEventExecutor.INSTANCE).setFailure(t);
}
// as the Channel is not registered yet we need to force the usage of the GlobalEventExecutor
return new DefaultChannelPromise(new FailedChannel(), GlobalEventExecutor.INSTANCE).setFailure(t);
}
ChannelFuture regFuture = config().group().register(channel);
if (regFuture.cause() != null) {
if (channel.isRegistered()) {
channel.close();
} else {
channel.unsafe().closeForcibly();
}
}
// If we are here and the promise is not failed, it's one of the following cases:
// 1) If we attempted registration from the event loop, the registration has been completed at this point.
// i.e. It's safe to attempt bind() or connect() now because the channel has been registered.
// 2) If we attempted registration from the other thread, the registration request has been successfully
// added to the event loop's task queue for later execution.
// i.e. It's safe to attempt bind() or connect() now:
// because bind() or connect() will be executed *after* the scheduled registration task is executed
// because register(), bind(), and connect() are all bound to the same thread.
return regFuture;这里主要看初始化方法
@Override
void init(Channel channel) {
setChannelOptions(channel, newOptionsArray(), logger);
setAttributes(channel, attrs0().entrySet().toArray(EMPTY_ATTRIBUTE_ARRAY));
//构建pipeline
ChannelPipeline p = channel.pipeline();
final EventLoopGroup currentChildGroup = childGroup;
final ChannelHandler currentChildHandler = childHandler;
final Entry<ChannelOption<?>, Object>[] currentChildOptions;
synchronized (childOptions) {
currentChildOptions = childOptions.entrySet().toArray(EMPTY_OPTION_ARRAY);
}
final Entry<AttributeKey<?>, Object>[] currentChildAttrs = childAttrs.entrySet().toArray(EMPTY_ATTRIBUTE_ARRAY);
//channelInitializer一次性，初始化handler
//添加一个ServerBootstrapAcceptor，创建完后就移除了
//ServerBootstrapAcceptor，负责接收到连接之后创建连接后对连接的初始化工作
p.addLast(new ChannelInitializer<Channel>() {
@Override
public void initChannel(final Channel ch) {
final ChannelPipeline pipeline = ch.pipeline();
ChannelHandler handler = config.handler();
if (handler != null) {
pipeline.addLast(handler);
}
ch.eventLoop().execute(new Runnable() {
@Override
public void run() {
pipeline.addLast(new ServerBootstrapAcceptor(
ch, currentChildGroup, currentChildHandler, currentChildOptions, currentChildAttrs));
}
});
}
});
}经过这个方法后，开始了注册channel的工作
//注册channel
ChannelFuture regFuture = config().group().register(channel);我们断点进去，一路往下走来到了io.netty.channel.AbstractChannel.AbstractUnsafe#register这个方法
@Override
public final void register(EventLoop eventLoop, final ChannelPromise promise) {
ObjectUtil.checkNotNull(eventLoop, "eventLoop");
if (isRegistered()) {
promise.setFailure(new IllegalStateException("registered to an event loop already"));
return;
}
if (!isCompatible(eventLoop)) {
promise.setFailure(
new IllegalStateException("incompatible event loop type: " + eventLoop.getClass().getName()));
return;
}
AbstractChannel.this.eventLoop = eventLoop;
//判断是不是在EventLoop中的线程
if (eventLoop.inEventLoop()) {
register0(promise);
} else {
try {
eventLoop.execute(new Runnable() {
@Override
public void run() {
register0(promise);
}
});
} catch (Throwable t) {
logger.warn(
"Force-closing a channel whose registration task was not accepted by an event loop: {}",
AbstractChannel.this, t);
closeForcibly();
closeFuture.setClosed();
safeSetFailure(promise, t);
}
}
}由于现在还是主线程在工作，所以到了else里，看到有一个register0方法被封装成了一个task扔进了执行器里，先断点跟到execute方法里去
private void execute(Runnable task, boolean immediate) {
boolean inEventLoop = inEventLoop();
//将task添加到EventLoopGroup里
addTask(task);
if (!inEventLoop) {
//启动一个线程
startThread();
if (isShutdown()) {
boolean reject = false;
try {
if (removeTask(task)) {
reject = true;
}
} catch (UnsupportedOperationException e) {
// The task queue does not support removal so the best thing we can do is to just move on and
// hope we will be able to pick-up the task before its completely terminated.
// In worst case we will log on termination.
}
if (reject) {
reject();
}
}
}
if (!addTaskWakesUp && immediate) {
wakeup(inEventLoop);
}
}看看startThread方法
private void startThread() {
//首先判断线程是否是启动状态的，很显然现在是没启动的状态
if (state == ST_NOT_STARTED) {
//注意compareAndSet，CAS原子更新状态为启动状态，这里就是无锁实现并发了
if (STATE_UPDATER.compareAndSet(this, ST_NOT_STARTED, ST_STARTED)) {
boolean success = false;
try {
//这里才是真正的启动方法，是不是感觉很熟悉，spring中真正做操作的方法也是以do来开头的，前面都是铺垫
doStartThread();
success = true;
} finally {
if (!success) {
STATE_UPDATER.compareAndSet(this, ST_STARTED, ST_NOT_STARTED);
}
}
}
}
}线程启动完，执行regist0方法
private void register0(ChannelPromise promise) {
try {
// check if the channel is still open as it could be closed in the mean time when the register
// call was outside of the eventLoop
if (!promise.setUncancellable() || !ensureOpen(promise)) {
return;
}
boolean firstRegistration = neverRegistered;
//真正执行register的方法
doRegister();
neverRegistered = false;
registered = true;
// Ensure we call handlerAdded(...) before we actually notify the promise. This is needed as the
// user may already fire events through the pipeline in the ChannelFutureListener.
pipeline.invokeHandlerAddedIfNeeded();
//操作成功异步通知到regFuture
safeSetSuccess(promise);
pipeline.fireChannelRegistered();
// Only fire a channelActive if the channel has never been registered. This prevents firing
// multiple channel actives if the channel is deregistered and re-registered.
if (isActive()) {
if (firstRegistration) {
pipeline.fireChannelActive();
} else if (config().isAutoRead()) {
// This channel was registered before and autoRead() is set. This means we need to begin read
// again so that we process inbound data.
//
// See https://github.com/netty/netty/issues/4805
beginRead();
}
}
} catch (Throwable t) {
// Close the channel directly to avoid FD leak.
closeForcibly();
closeFuture.setClosed();
safeSetFailure(promise, t);
}
}进入到doRegister方法
@Override
protected void doRegister() throws Exception {
boolean selected = false;
for (;;) {
try {
//这里的selector就是NIOEventLoop里绑定的Selector
selectionKey = javaChannel().register(eventLoop().unwrappedSelector(), 0, this);
return;
} catch (CancelledKeyException e) {
if (!selected) {
// Force the Selector to select now as the "canceled" SelectionKey may still be
// cached and not removed because no Select.select(..) operation was called yet.
eventLoop().selectNow();
selected = true;
} else {
// We forced a select operation on the selector before but the SelectionKey is still cached
// for whatever reason. JDK bug ?
throw e;
}
}
}
}走完其余的部分，来到了io.netty.bootstrap.AbstractBootstrap#doBind0方法，断点跟进去
private static void doBind0(
final ChannelFuture regFuture, final Channel channel,
final SocketAddress localAddress, final ChannelPromise promise) {
// This method is invoked before channelRegistered() is triggered. Give user handlers a chance to set up
// the pipeline in its channelRegistered() implementation.
channel.eventLoop().execute(new Runnable() {
@Override
public void run() {
if (regFuture.isSuccess()) {
//channel绑定
channel.bind(localAddress, promise).addListener(ChannelFutureListener.CLOSE_ON_FAILURE);
} else {
promise.setFailure(regFuture.cause());
}
}
});
}继续进入bind方法
@Override
public ChannelFuture bind(SocketAddress localAddress, ChannelPromise promise) {
//pipeline
return pipeline.bind(localAddress, promise);
}在这里pipeline里其实有很多的handler，要是全部跟源码的话要走很多的handler，所以我在这里就挑主要的headContext跟进，来到了io.netty.channel.DefaultChannelPipeline.HeadContext#bind方法
@Override
public void bind(
ChannelHandlerContext ctx, SocketAddress localAddress, ChannelPromise promise) {
unsafe.bind(localAddress, promise);
}
@Override
public final void bind(final SocketAddress localAddress, final ChannelPromise promise) {
assertEventLoop();
if (!promise.setUncancellable() || !ensureOpen(promise)) {
return;
}
// See: https://github.com/netty/netty/issues/576
if (Boolean.TRUE.equals(config().getOption(ChannelOption.SO_BROADCAST)) &&
localAddress instanceof InetSocketAddress &&
!((InetSocketAddress) localAddress).getAddress().isAnyLocalAddress() &&
!PlatformDependent.isWindows() && !PlatformDependent.maybeSuperUser()) {
// Warn a user about the fact that a non-root user can't receive a
// broadcast packet on *nix if the socket is bound on non-wildcard address.
logger.warn(
"A non-root user can't receive a broadcast packet if the socket " +
"is not bound to a wildcard address; binding to a non-wildcard " +
"address (" + localAddress + ") anyway as requested.");
}
boolean wasActive = isActive();
try {
//真正bind的方法
doBind(localAddress);
} catch (Throwable t) {
safeSetFailure(promise, t);
closeIfClosed();
return;
}
if (!wasActive && isActive()) {
invokeLater(new Runnable() {
@Override
public void run() {
pipeline.fireChannelActive();
}
});
}
safeSetSuccess(promise);
}进入doBind方法，来到io.netty.channel.socket.nio.NioServerSocketChannel#doBind
@SuppressJava6Requirement(reason = "Usage guarded by java version check")
@Override
protected void doBind(SocketAddress localAddress) throws Exception {
//由于我用的是jdk1.8，所以会走上面这一行
if (PlatformDependent.javaVersion() >= 7) {
javaChannel().bind(localAddress, config.getBacklog());
} else {
javaChannel().socket().bind(localAddress, config.getBacklog());
}
}走完bind方法之后，回到这段代码
if (!wasActive && isActive()) {
invokeLater(new Runnable() {
@Override
public void run() {
//这也是个pipeline，我们看看fireChannelActive做了什么
pipeline.fireChannelActive();
}
});
}老样子，跳到headContext里面，找到channelActive
@Override
public void channelActive(ChannelHandlerContext ctx) {
ctx.fireChannelActive();
//注册读事件，读事件包括创建连接或者读数据，重点是这行，这里我们是创建连接
readIfIsAutoRead();
}跟进readIfIsAutoRead方法
private void readIfIsAutoRead() {
if (channel.config().isAutoRead()) {
//如果配置的是autoRead，就开始读数据
channel.read();
}
}再跟进
@Override
public Channel read() {
pipeline.read();
return this;
}老样子，找到DefaultChannelPipeline的read方法
@Override
public void read(ChannelHandlerContext ctx) {
unsafe.beginRead();
}再跟进
@Override
public final void beginRead() {
assertEventLoop();
if (!isActive()) {
return;
}
try {
//真正开始beginRead
doBeginRead();
} catch (final Exception e) {
invokeLater(new Runnable() {
@Override
public void run() {
pipeline.fireExceptionCaught(e);
}
});
close(voidPromise());
}
}一路往下跟进，最后来到了io.netty.channel.nio.AbstractNioChannel#doBeginRead方法
@Override
protected void doBeginRead() throws Exception {
// Channel.read() or ChannelHandlerContext.read() was called
final SelectionKey selectionKey = this.selectionKey;
if (!selectionKey.isValid()) {
return;
}
readPending = true;
final int interestOps = selectionKey.interestOps();
//如果之前没有监听readInterestOp，则开始监听
if ((interestOps & readInterestOp) == 0) {
selectionKey.interestOps(interestOps | readInterestOp);
}
}走到了这里，表示服务已经启动成功，并且准备处理创建连接事件。
总结
我们通过分析源码阅读源码，现在了解了Netty启动服务的整个过程，我用一幅图来总结一下这个流程
2102
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
