手把手教你学习netty源码及原理
本文通过netty的简单例子，从源码视角分析netty工作原理。netty是基于reactor的高性能网络nio框架，对nio的阻塞、异步、reactor模式不熟悉的同学可以参考上一篇的博文 https://blog.csdn.net/Houson_c/article/details/86114771。
netty的核心组件
channel：对应jdkchannel的抽象，还有其他实现类如epollniochannel，代表一个socket连接的
channelpipeline：是事件处理管道，channel的register、连接、读写事件的在pipeline中流通，被channelhandler拦截处理
channelhandler：处理channel事件的逻辑实现，可以用来做解码、编码、业务逻辑处理
NioEventLoop/NioEventLoopGroup：线程池抽象，包含worker线程，用来执行io事件handler的代码和用户task、定时任务等，一个channel对应一个eventloop避免多线程竞争。
netty使用实践
先来看一个简单的netty的客户端和服务端的实例
服务端代码：
这个是一个服务端将当前时间返回给客户端的简单实例，当客户端连接到来时将当前时间直接返回。
public class NettyServer {
private int port;
public NettyServer(int port) {
this.port = port;
}
public void run() throws Exception {
// (1)创建主从reactor,线程池
EventLoopGroup bossGroup = new NioEventLoopGroup(1);
EventLoopGroup workerGroup = new NioEventLoopGroup(1);
try {
// (2)启动器类
ServerBootstrap b = new ServerBootstrap();
b.group(bossGroup, workerGroup)
// (3)指定io类型，epoll，udp，oio,kqueue
.channel(NioServerSocketChannel.class)
// (4)创建Inithandler
.childHandler(new ChannelInitializer<SocketChannel>() {
@Override
public void initChannel(SocketChannel ch) throws Exception {
//（5）定义pipeline处理过程,运行在的worker线程池
ch.pipeline().addLast("ahandler",new TimeServerHandler());
}
})
// (6)配置main reactor属性，影响连接建立例如socket连接数等
.option(ChannelOption.SO_BACKLOG, 128)
//(7)配置subreactor的属性，影响io处理
.childOption(ChannelOption.SO_KEEPALIVE, true);
// (8)Bind and start to accept incoming connections.
ChannelFuture f = b.bind(port).sync();
f.channel().closeFuture().sync();
} finally {
workerGroup.shutdownGracefully();
bossGroup.shutdownGracefully();
}
}
/**
* 基于事件驱动的handler
*/
public static class TimeServerHandler extends ChannelInboundHandlerAdapter {
//(1)新连接到来
@Override
public void channelActive(final ChannelHandlerContext ctx) {
final ByteBuf time = ctx.alloc().buffer(4);
time.writeInt((int) (System.currentTimeMillis() / 1000L + 2208988800L));
// (2)回写时间数据
final ChannelFuture f = ctx.writeAndFlush(time);
f.addListener(new ChannelFutureListener() {
@Override
public void operationComplete(ChannelFuture future) {
assert f == future;
ctx.close();
}
});
}
//(3)接受客户端数据
@Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
super.channelRead(ctx, msg);
}
//读取完成事件，一般不用
@Override
public void channelReadComplete(ChannelHandlerContext ctx) throws Exception {
super.channelReadComplete(ctx);
}
//发生异常事件触发
@Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
cause.printStackTrace();
ctx.close();
}
}
}
public static void main(String[] args) throws Exception {
int port = 6666;
if (args.length > 0) {
port = Integer.parseInt(args[0]);
}
new NettyServer(port).run();
}
简单分析上面的服务端代码，
(1)创建主从reactor,线程池，这里为了方便调试，设置线程池线程数量都为1
(2)创建启动器类对象，netty的辅助类
(3)指定io类型，epoll，udp，oio,kqueue
(4)创建Inithandler
(5)定义pipeline处理过程,运行在的worker线程池
(6)配置main reactor属性，影响连接建立例如socket连接数等
(7)配置subreactor的属性，影响io处理
(8)Bind and start to accept incoming connections.开始接受客户端连接，进行初始化。
客户端代码:
客户端连接到服务端之后，通过handler直接打印服务端返回的结果。
public class NettyClient {
public static void main(String[] args) throws InterruptedException {
String host = "localhost";
int port = 6666;
EventLoopGroup workerGroup = new NioEventLoopGroup();
try {
Bootstrap b = new Bootstrap(); // (1)
b.group(workerGroup); // (2)
b.channel(NioSocketChannel.class); // (3)
b.option(ChannelOption.SO_KEEPALIVE, true); // (4)
b.handler(new ChannelInitializer<SocketChannel>() {
@Override
public void initChannel(SocketChannel ch) throws Exception {
ch.pipeline().addLast(new TimeClientHandler());
}
});
// Start the client.
ChannelFuture f = b.connect(host, port).sync(); // (5)
// Wait until the connection is closed.
f.channel().closeFuture().sync();
} finally {
workerGroup.shutdownGracefully();
}
}
public static class TimeClientHandler extends ChannelInboundHandlerAdapter {
@Override
public void channelRead(ChannelHandlerContext ctx, Object msg) {
ByteBuf m = (ByteBuf) msg; // (1)
try {
//打印结果
long currentTimeMillis = (m.readUnsignedInt() - 2208988800L) * 1000L;
System.out.println(new Date(currentTimeMillis));
ctx.close();
} finally {
m.release();
}
}
@Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
cause.printStackTrace();
ctx.close();
}
}
}
上述代码可以直接运行可以看到运行结果为，客户端输出：
Sat Feb 09 22:25:01 CST 2019
netty源码分析
基于上面的例子，来分析一下netty重初始化到客户端连接建立在到完成读写io操作的流程。
源码环境搭建
-
下载netty的源码，如果使用的是maven依赖可以直接通过maven的导入源码和文档
-
添加maven依赖
<dependency> <groupId>io.netty</groupId> <artifactId>netty-all</artifactId> <version>4.1.30.Final</version> </dependency> -
下载源码和文档
-
-
以调试模式起启动NettyServer服务端程序，注意idea中使用断点是设置Thread，方便选择线程进行调试。因为boss线程和worker线程都是在不停的运行不选择线程调试容易迷失，代码中只用了一个boss和一个worker线程。
NettyServer启动初始化流程
从NettyServer的例子的代码来看
// (1)创建主从reactor,线程池
EventLoopGroup bossGroup = new NioEventLoopGroup(1);
EventLoopGroup workerGroup = new NioEventLoopGroup(1);
首先，创建NioEventLoopGroup，EventLoopGroup是线程池的封装。
从继承关系看出NioEventLopGroup是基于javanio的实现，还有epoll、Kqueue的实现。
NioEventLoopGroup的构造方法直接调用类父类MultithreadEventLoopGroup的构造方法，代码如下
protected MultithreadEventLoopGroup(int nThreads, Executor executor, Object... args) {
//默认是DEFAULT_EVENT_LOOP_THREADS = NettyRuntime.availableProcessors() * 2
super(nThreads == 0 ? DEFAULT_EVENT_LOOP_THREADS : nThreads, executor, args);
}
protected MultithreadEventExecutorGroup(int nThreads, Executor executor,
EventExecutorChooserFactory chooserFactory, Object... args) {
if (nThreads <= 0) {
throw new IllegalArgumentException(String.format("nThreads: %d (expected: > 0)", nThreads));
}
if (executor == null) {
//每个executor都是线程池
executor = new ThreadPerTaskExecutor(newDefaultThreadFactory());
}
//创建EventLoop数组
children = new EventExecutor[nThreads];
for (int i = 0; i < nThreads; i ++) {
boolean success = false;
try {
//创建NioEventLoop，每个EventLoop创建一个线程
children[i] = newChild(executor, args);
success = true;
} catch (Exception e) {
// TODO: Think about if this is a good exception type
throw new IllegalStateException("failed to create a child event loop", e);
} finally {
//容错处理，省略。。。
}
}
//创建Eventloop选择器，用于选择一个NioEventloop进行io事件处理
chooser = chooserFactory.newChooser(children);
//省略。。。
}
主要做了这几步：
- DEFAULT_EVENT_LOOP_THREADS = NettyRuntime.availableProcessors() * 2
- 创建EventLoop数组 children = new EventExecutor[nThreads];
- 创建NioEventLoop，每个EventLoop创建一个线程， children[i] = newChild(executor, args);
- 创建Eventloop选择器，用于选择一个NioEventloop进行io事件处理
bossgroup和workergroup都是这样的创建流程。上面几步中，重点看一下NioEventLoop的创建和初始化过程。
NioEventLoop(NioEventLoopGroup parent, Executor executor, SelectorProvider selectorProvider,
SelectStrategy strategy, RejectedExecutionHandler rejectedExecutionHandler) {
super(parent, executor, false, DEFAULT_MAX_PENDING_TASKS, rejectedExecutionHandler);
if (selectorProvider == null) {
throw new NullPointerException("selectorProvider");
}
if (strategy == null) {
throw new NullPointerException("selectStrategy");
}
provider = selectorProvider;
//每个NioEventLoop都创建selector
final SelectorTuple selectorTuple = openSelector();
selector = selectorTuple.selector;
unwrappedSelector = selectorTuple.unwrappedSelector;
selectStrategy = strategy;
}
既然每个eventloop是一个线程那么他的run方法肯定是eventloop的核心逻辑，看一下他的实现
@Override
protected void run() {
for (;;) {
try {
//判断任务队列中是否有任务待执行的任务
switch (selectStrategy.calculateStrategy(selectNowSupplier, hasTasks())) {
case SelectStrategy.CONTINUE:
continue;
case SelectStrategy.SELECT:
select(wakenUp.getAndSet(false));
if (wakenUp.get()) {
selector.wakeup();
}
// fall through
default:
}
cancelledKeys = 0;
needsToSelectAgain = false;
final int ioRatio = this.ioRatio;
if (ioRatio == 100) {
try {
processSelectedKeys();
} finally {
// Ensure we always run tasks.
runAllTasks();
}
} else {
final long ioStartTime = System.nanoTime();
try {
processSelectedKeys();
} finally {
// Ensure we always run tasks.
final long ioTime = System.nanoTime() - ioStartTime;
runAllTasks(ioTime * (100 - ioRatio) / ioRatio);
}
}
} catch (Throwable t) {
handleLoopException(t);
}
// Always handle shutdown even if the loop processing threw an exception.
try {
if (isShuttingDown()) {
closeAll();
if (confirmShutdown()) {
return;
}
}
} catch (Throwable t) {
handleLoopException(t);
}
}
}
run方法中无限循环selector监听是否有io事件，最终会进入下面的代码执行io事件处理
private void processSelectedKey(SelectionKey k, AbstractNioChannel ch) {
final AbstractNioChannel.NioUnsafe unsafe = ch.unsafe();
if (!k.isValid()) {
final EventLoop eventLoop;
try {
eventLoop = ch.eventLoop();
} catch (Throwable ignored) {
return;
}
if (eventLoop != this || eventLoop == null) {
return;
}
// close the channel if the key is not valid anymore
unsafe.close(unsafe.voidPromise());
return;
}
try {
int readyOps = k.readyOps();
if ((readyOps & SelectionKey.OP_CONNECT) != 0) {
// remove OP_CONNECT as otherwise Selector.select(..) will always return without blocking
// See https://github.com/netty/netty/issues/924
int ops = k.interestOps();
ops &= ~SelectionKey.OP_CONNECT;
k.interestOps(ops);
unsafe.finishConnect();
}
// Process OP_WRITE first as we may be able to write some queued buffers and so free memory.
if ((readyOps & SelectionKey.OP_WRITE) != 0) {
// Call forceFlush which will also take care of clear the OP_WRITE once there is nothing left to write
ch.unsafe().forceFlush();
}
// Also check for readOps of 0 to workaround possible JDK bug which may otherwise lead
// to a spin loop
if ((readyOps & (SelectionKey.OP_READ | SelectionKey.OP_ACCEPT)) != 0 || readyOps == 0) {
unsafe.read();
}
} catch (CancelledKeyException ignored) {
unsafe.close(unsafe.voidPromise());
}
}
这个方法根据selectionKey不同的的事件最终调用Unsafe的对应方法进行处理，Unsafe是netty内部的辅助类，开发者不应该直接调用该类的方法所以叫unsafe。
这就是EventLoopGroup的初始化过程，先不看具体的事件处理逻辑，继续看NettyServer启动的初始化过程的代码，从上面的例子看出，server启动是在bind()方法开始的，看下具体的实现。
private ChannelFuture doBind(final SocketAddress localAddress) {
//初始化和注册serverSocket到Selector
final ChannelFuture regFuture = initAndRegister();
final Channel channel = regFuture.channel();
if (regFuture.cause() != null) {
return regFuture;
}
if (regFuture.isDone()) {
// At this point we know that the registration was complete and successful.
ChannelPromise promise = channel.newPromise();
//执行bind
doBind0(regFuture, channel, localAddress, promise);
return promise;
} else {
// Registration future is almost always fulfilled already, but just in case it's not.
final PendingRegistrationPromise promise = new PendingRegistrationPromise(channel);
regFuture.addListener(new ChannelFutureListener() {
@Override
public void operationComplete(ChannelFuture future) throws Exception {
Throwable cause = future.cause();
if (cause != null) {
// Registration on the EventLoop failed so fail the ChannelPromise directly to not cause an
// IllegalStateException once we try to access the EventLoop of the Channel.
promise.setFailure(cause);
} else {
// Registration was successful, so set the correct executor to use.
// See https://github.com/netty/netty/issues/2586
promise.registered();
doBind0(regFuture, channel, localAddress, promise);
}
}
});
return promise;
}
}
AbstractBootstrap的dobind方法中进行了initAndRegister将ServerScoektChannel初始化和注册到Selector，然后执行了dobind0执行bind操作，先看initAndRegister
final ChannelFuture initAndRegister() {
Channel channel = null;
try {
//默认通过反射创建Serverchannel，NioServerScoketChanel
channel = channelFactory.newChannel();
//初始化ServerChannel，添加Acceptor到pipeline
init(channel);
} catch (Throwable t) {
//......
}
//注册到selector
ChannelFuture regFuture = config().group().register(channel);
if (regFuture.cause() != null) {
if (channel.isRegistered()) {
channel.close();
} else {
channel.unsafe().closeForcibly();
}
}
//......
return regFuture;
}
serverchannel的init
@Override
void init(Channel channel) throws Exception {
//配置属性。。。。。
p.addLast(new ChannelInitializer<Channel>() {
@Override
public void initChannel(final Channel ch) throws Exception {
final ChannelPipeline pipeline = ch.pipeline();
ChannelHandler handler = config.handler();
if (handler != null) {
//添加配置的handler
pipeline.addLast(handler);
}
ch.eventLoop().execute(new Runnable() {
@Override
public void run() {
//添加Acceptor到ServerSocketChannel的pipline
pipeline.addLast(new ServerBootstrapAcceptor(
ch, currentChildGroup, currentChildHandler, currentChildOptions, currentChildAttrs));
}
});
}
});
}
ServerChannel的register最终会调用到NioEventLoop的Register方法
@Override
public ChannelFuture register(final ChannelPromise promise) {
ObjectUtil.checkNotNull(promise, "promise");
promise.channel().unsafe().register(this, promise);
return promise;
}
Channel的大部分操作都是调用的Unsafe的方法实现的，最终会调用AbstractUnsafe的register0进行注册。
private void register0(ChannelPromise promise) {
try {
// check if the channel is still open as it could be closed in the mean time when the register
// call was outside of the eventLoop
if (!promise.setUncancellable() || !ensureOpen(promise)) {
return;
}
boolean firstRegistration = neverRegistered;
//将channel注册到selector，最终是通过jdk的nio api实现的（nio实现下）
doRegister();
neverRegistered = false;
registered = true;
// Ensure we call handlerAdded(...) before we actually notify the promise. This is needed as the
// user may already fire events through the pipeline in the ChannelFutureListener.
//添加用户定义的事件Handler对象
pipeline.invokeHandlerAddedIfNeeded();
safeSetSuccess(promise);
//触发channelRegister事件，通过pipeline进行触发，并通过handler链进行传播
pipeline.fireChannelRegistered();
// Only fire a channelActive if the channel has never been registered. This prevents firing
// multiple channel actives if the channel is deregistered and re-registered.
if (isActive()) {
if (firstRegistration) {
//触发channelActive事件，通过pipeline进行触发，并通过handler链进行传播
pipeline.fireChannelActive();
} else if (config().isAutoRead()) {
// This channel was registered before and autoRead() is set. This means we need to begin read
// again so that we process inbound data.
//
// See https://github.com/netty/netty/issues/4805
//注册读事件监听到selector
beginRead();
}
}
} catch (Throwable t) {
// Close the channel directly to avoid FD leak.
closeForcibly();
closeFuture.setClosed();
safeSetFailure(promise, t);
}
}
register0调用doregister最终调用jdk的register方法进行channel的注册到selector，之后通过 pipeline.invokeHandlerAddedIfNeeded();将用户定义的编码器、解码器、inboundhandler等添加到pipeline中，最后，在pipeline中触发channelRegister和channelActive事件。
回到dobind方法，在initRegister完成之后，会调用dobind0方法，进行ServerSocket的bind，如下
private static void doBind0(
final ChannelFuture regFuture, final Channel channel,
final SocketAddress localAddress, final ChannelPromise promise) {
// This method is invoked before channelRegistered() is triggered. Give user handlers a chance to set up
// the pipeline in its channelRegistered() implementation.
channel.eventLoop().execute(new Runnable() {
@Override
public void run() {
if (regFuture.isSuccess()) {
channel.bind(localAddress, promise).addListener(ChannelFutureListener.CLOSE_ON_FAILURE);
} else {
promise.setFailure(regFuture.cause());
}
}
});
}
调用channel的bind方法进行bind操作，并且是在channel的eventloop中执行，最终是调用了pipeline的bind方法进行bind操作，在pipeline 链中的调用流程为：
pipeline.bind->tail.bind->chennelHandlerContext.bind->head.bind->unsafe.bind->javaChannel.bind
@Override
public final void bind(final SocketAddress localAddress, final ChannelPromise promise) {
assertEventLoop();
if (!promise.setUncancellable() || !ensureOpen(promise)) {
return;
}
// See: https://github.com/netty/netty/issues/576
if (Boolean.TRUE.equals(config().getOption(ChannelOption.SO_BROADCAST)) &&
localAddress instanceof InetSocketAddress &&
!((InetSocketAddress) localAddress).getAddress().isAnyLocalAddress() &&
!PlatformDependent.isWindows() && !PlatformDependent.maybeSuperUser()) {
// Warn a user about the fact that a non-root user can't receive a
// broadcast packet on *nix if the socket is bound on non-wildcard address.
logger.warn(
"A non-root user can't receive a broadcast packet if the socket " +
"is not bound to a wildcard address; binding to a non-wildcard " +
"address (" + localAddress + ") anyway as requested.");
}
boolean wasActive = isActive();
try {
doBind(localAddress);
} catch (Throwable t) {
safeSetFailure(promise, t);
closeIfClosed();
return;
}
if (!wasActive && isActive()) {
invokeLater(new Runnable() {
@Override
public void run() {
//触发active事件
pipeline.fireChannelActive();
}
});
}
safeSetSuccess(promise);
}
其中tail为TailContext，head为HeadContext，是netty在pipline的首尾自动添加的Handler，可以看到最终调用的式jdk的channel.bind方法。
bind完成后通过 pipeline.fireChannelActive();在pipline中发出channelActive事件，pipeline的链式处理流程为
DefaultChannelPipeline.fireChannelActive
->AbstractChannelHandlerContext.invokeChannelActive
->head.invokeChannelActive
->自定义的TimeServerHandler.channelActive
->tail.channelActive
小结：
Server的启动流程关键节点总结如下：
- 创建EventLoppGroup，创建EventLopp，每个eventLoop创建selector，启动select循环
- 创建ServerChannel，创建对应的pipeline，创建Acceptor并添加到pipeline，
- 将ServerChannel注册到Selector，监听事件操作为0
- 添加初始化的事件处理Handler
- pipeline触发channelRegister事件
- 执行bind
- pipeline触发channelActive事件
初始化流程剔除掉一些细节其实和上一篇文章的server启动主流程基本是一致的，多了一些pipeline的步骤，这也是netty的优势所在，这里面涉及的各个组件如果不熟悉的童鞋可以参考,后续也会继续推出相关blog
https://waylau.gitbooks.io/essential-netty-in-action/
客户端建立连接
在Server创建和初始化之后NioEventLoop的select循环已经启动了，当客户端socket连接到来时操作系统层面会发出连接事件，Severchannel会监听该事件并创建客户端对应的NioSocketChannel连接，再看一下select循环的实现。
@Override
protected void run() {
for (;;) {
try {
switch (selectStrategy.calculateStrategy(selectNowSupplier, hasTasks())) {
case SelectStrategy.CONTINUE:
continue;
case SelectStrategy.SELECT:
select(wakenUp.getAndSet(false));
if (wakenUp.get()) {
selector.wakeup();
}
// fall through
default:
}
cancelledKeys = 0;
needsToSelectAgain = false;
final int ioRatio = this.ioRatio;
if (ioRatio == 100) {
try {
//处理selectionKey的事件
processSelectedKeys();
} finally {
// Ensure we always run tasks.
runAllTasks();
}
} else {
final long ioStartTime = System.nanoTime();
try {
processSelectedKeys();
} finally {
// Ensure we always run tasks.
//根据iorate确定执行非io任务的事件
final long ioTime = System.nanoTime() - ioStartTime;
runAllTasks(ioTime * (100 - ioRatio) / ioRatio);
}
}
} catch (Throwable t) {
handleLoopException(t);
}
}
}
很明显，processSelectedKeys处理了io事件，netty对jdk的SelectedKeys进行了优化会调用，processSelectedKeysOptimized方法，最终到下面的代码
private void processSelectedKey(SelectionKey k, AbstractNioChannel ch) {
final AbstractNioChannel.NioUnsafe unsafe = ch.unsafe();
if (!k.isValid()) {
final EventLoop eventLoop;
try {
eventLoop = ch.eventLoop();
} catch (Throwable ignored) {
return;
}
if (eventLoop != this || eventLoop == null) {
return;
}
// close the channel if the key is not valid anymore
unsafe.close(unsafe.voidPromise());
return;
}
try {
int readyOps = k.readyOps();
// We first need to call finishConnect() before try to trigger a read(...) or write(...) as otherwise
// the NIO JDK channel implementation may throw a NotYetConnectedException.
if ((readyOps & SelectionKey.OP_CONNECT) != 0) {
// remove OP_CONNECT as otherwise Selector.select(..) will always return without blocking
// See https://github.com/netty/netty/issues/924
int ops = k.interestOps();
ops &= ~SelectionKey.OP_CONNECT;
k.interestOps(ops);
unsafe.finishConnect();
}
// Process OP_WRITE first as we may be able to write some queued buffers and so free memory.
if ((readyOps & SelectionKey.OP_WRITE) != 0) {
// Call forceFlush which will also take care of clear the OP_WRITE once there is nothing left to write
ch.unsafe().forceFlush();
}
// Also check for readOps of 0 to workaround possible JDK bug which may otherwise lead
// to a spin loop
if ((readyOps & (SelectionKey.OP_READ | SelectionKey.OP_ACCEPT)) != 0 || readyOps == 0) {
unsafe.read();
}
} catch (CancelledKeyException ignored) {
unsafe.close(unsafe.voidPromise());
}
}
可以看到这里对SelectionKey.OP_WRITE,SelectionKey.OP_READ,SelectionKey.OP_ACCEPT进行了判断，我们来看SelectionKey.OP_ACCEPT的处理，调用了unsafe.read()方法进行处理
最终会调用到NioServerSocketChannel的readMessages方法
@Override
protected int doReadMessages(List<Object> buf) throws Exception {
//accept客户端连接
SocketChannel ch = SocketUtils.accept(javaChannel());
try {
if (ch != null) {
//创建客户端的channel，把channel返回给ServerBootstrapAcceptor处理
buf.add(new NioSocketChannel(this, ch));
return 1;
}
} catch (Throwable t) {
logger.warn("Failed to create a new channel from an accepted socket.", t);
try {
ch.close();
} catch (Throwable t2) {
logger.warn("Failed to close a socket.", t2);
}
}
return 0;
}
上述方法在把NioSocketChannel放在buf中返回，然后在pipeline中fireread，而上面server启动初始化时创建了ServerBootstrapAcceptor并将其添加到了pipeline，最终会调用ServerBootstrapAcceptor的channelRead（）
@Override
@SuppressWarnings("unchecked")
public void channelRead(ChannelHandlerContext ctx, Object msg) {
final Channel child = (Channel) msg;
//给客户端channel添加用户定义的handler
child.pipeline().addLast(childHandler);
setChannelOptions(child, childOptions, logger);
for (Entry<AttributeKey<?>, Object> e: childAttrs) {
child.attr((AttributeKey<Object>) e.getKey()).set(e.getValue());
}
try {
//注册Read监听，和ServerChannel的register类似的流程
childGroup.register(child).addListener(new ChannelFutureListener() {
@Override
public void operationComplete(ChannelFuture future) throws Exception {
if (!future.isSuccess()) {
forceClose(child, future.cause());
}
}
});
} catch (Throwable t) {
forceClose(child, t);
}
}
创建完客户端channel之后会从到worker线程池的选择一个eventloop，将channel注册该eventloop的selector上进行read事件监听，这里的register步骤和serverchannel的register类似，不过register0是在worker线程池的eventloop执行了。这里开始连接就建立完成了，调试代码时，这里应该要切换成worker线程进行调试了。
连接流程小结：
- select循环监听accept事件
- SocketUtils.accept建立连接创建客户端SocketChannel
- Acceptor从childGroup中选择一个eventloop把客户端的NioSocketChannel register到eventloop的selector中，eventloop的select循环监听客户端channel的io事件
处理读事件
由上可以知道，客户端channel建立连接之后所有的操作都在WorkerEvenloopGroup中的NioEventLoop中进行了，再次看NioEventLoop的的run方法（第三次）。
private void processSelectedKey(SelectionKey k, AbstractNioChannel ch) {
final AbstractNioChannel.NioUnsafe unsafe = ch.unsafe();
if (!k.isValid()) {
final EventLoop eventLoop;
try {
eventLoop = ch.eventLoop();
} catch (Throwable ignored) {
return;
}
if (eventLoop != this || eventLoop == null) {
return;
}
// close the channel if the key is not valid anymore
unsafe.close(unsafe.voidPromise());
return;
}
try {
int readyOps = k.readyOps();
// We first need to call finishConnect() before try to trigger a read(...) or write(...) as otherwise
// the NIO JDK channel implementation may throw a NotYetConnectedException.
if ((readyOps & SelectionKey.OP_CONNECT) != 0) {
// remove OP_CONNECT as otherwise Selector.select(..) will always return without blocking
// See https://github.com/netty/netty/issues/924
int ops = k.interestOps();
ops &= ~SelectionKey.OP_CONNECT;
k.interestOps(ops);
unsafe.finishConnect();
}
// Process OP_WRITE first as we may be able to write some queued buffers and so free memory.
if ((readyOps & SelectionKey.OP_WRITE) != 0) {
// Call forceFlush which will also take care of clear the OP_WRITE once there is nothing left to write
ch.unsafe().forceFlush();
}
// Also check for readOps of 0 to workaround possible JDK bug which may otherwise lead
// to a spin loop
if ((readyOps & (SelectionKey.OP_READ | SelectionKey.OP_ACCEPT)) != 0 || readyOps == 0) {
unsafe.read();
}
} catch (CancelledKeyException ignored) {
unsafe.close(unsafe.voidPromise());
}
}
workerEventloop的unsafe是AbstractNioByteChannel的子类，而BossGroup的AbstractNioMessageChannel子类，实现如下
@Override
public final void read() {
final ChannelConfig config = config();
if (shouldBreakReadReady(config)) {
clearReadPending();
return;
}
final ChannelPipeline pipeline = pipeline();
//使用配置的ByteBufAllocator进行内存肥培
final ByteBufAllocator allocator = config.getAllocator();
//内存分配辅助类
final RecvByteBufAllocator.Handle allocHandle = recvBufAllocHandle();
allocHandle.reset(config);
ByteBuf byteBuf = null;
boolean close = false;
try {
do {
//可以根据历史内存分配大小进行guess，推测分配的大小
byteBuf = allocHandle.allocate(allocator);
//从channel中读取butes，存在ByteBuf中
allocHandle.lastBytesRead(doReadBytes(byteBuf));
if (allocHandle.lastBytesRead() <= 0) {
// nothing was read. release the buffer.
byteBuf.release();
byteBuf = null;
//判断是否读完整个message数据
close = allocHandle.lastBytesRead() < 0;
if (close) {
// There is nothing left to read as we received an EOF.
readPending = false;
}
break;
}
allocHandle.incMessagesRead(1);
readPending = false;
//pipeline触发channelRead事件
pipeline.fireChannelRead(byteBuf);
byteBuf = null;
} while (allocHandle.continueReading());//循环读取直到没有数据为止
allocHandle.readComplete();
//pipeline触发fireChannelReadComplete事件
pipeline.fireChannelReadComplete();
if (close) {
closeOnRead(pipeline);
}
} catch (Throwable t) {
handleReadException(pipeline, byteBuf, t, close, allocHandle);
} finally {
// Check if there is a readPending which was not processed yet.
// This could be for two reasons:
// * The user called Channel.read() or ChannelHandlerContext.read() in channelRead(...) method
// * The user called Channel.read() or ChannelHandlerContext.read() in channelReadComplete(...) method
//
// See https://github.com/netty/netty/issues/2254
if (!readPending && !config.isAutoRead()) {
removeReadOp();
}
}
}
上面的流程比较简单，分配内存然后循环读取channel的数据，直到整个message读完，ByteBuf的内存分配和回收比较复杂，如果有兴趣可以参考netty权威指南，后续继续写这方面的blog。
每次读取都会触发channelRead，所以用户自定的handler会对一个message多次收到ChannelRead方法的回调，需要进行粘包处理，后续会详细介绍。读取完数据之后会触发channelReadCompelete方法
处理写事件
channel写数据，直接调用writeAndFlush，流程如下
pipeline.writeAndFlush(msg);->tail.writeAndFlush(msg);->abstractChannelContext.invokeWriteAndFlush->HeadContext.write->unsafe.write
最终是调用unsafe的write方法，
@Override
public final void write(Object msg, ChannelPromise promise) {
assertEventLoop();
ChannelOutboundBuffer outboundBuffer = this.outboundBuffer;
if (outboundBuffer == null) {
// If the outboundBuffer is null we know the channel was closed and so
// need to fail the future right away. If it is not null the handling of the rest
// will be done in flush0()
// See https://github.com/netty/netty/issues/2362
safeSetFailure(promise, WRITE_CLOSED_CHANNEL_EXCEPTION);
// release message now to prevent resource-leak
ReferenceCountUtil.release(msg);
return;
}
int size;
try {
msg = filterOutboundMessage(msg);
size = pipeline.estimatorHandle().size(msg);
if (size < 0) {
size = 0;
}
} catch (Throwable t) {
safeSetFailure(promise, t);
ReferenceCountUtil.release(msg);
return;
}
//将write的message放在outbuffer中，通过flush，写到channel
outboundBuffer.addMessage(msg, size, promise);
}
unsafe.write将message放在outbuffer中，通过flush，写到channel，看一下unsafe的flush
调用NioSocketChannel进行写入，然后调用jdk 的channel写入到socket
@Override
protected void doWrite(ChannelOutboundBuffer in) throws Exception {
SocketChannel ch = javaChannel();
int writeSpinCount = config().getWriteSpinCount();
do {
if (in.isEmpty()) {
// All written so clear OP_WRITE
clearOpWrite();
// Directly return here so incompleteWrite(...) is not called.
return;
}
// Ensure the pending writes are made of ByteBufs only.
int maxBytesPerGatheringWrite = ((NioSocketChannelConfig) config).getMaxBytesPerGatheringWrite();
ByteBuffer[] nioBuffers = in.nioBuffers(1024, maxBytesPerGatheringWrite);
int nioBufferCnt = in.nioBufferCount();
// Always us nioBuffers() to workaround data-corruption.
// See https://github.com/netty/netty/issues/2761
switch (nioBufferCnt) {
case 0:
// We have something else beside ByteBuffers to write so fallback to normal writes.
writeSpinCount -= doWrite0(in);
break;
case 1: {
// Only one ByteBuf so use non-gathering write
// Zero length buffers are not added to nioBuffers by ChannelOutboundBuffer, so there is no need
// to check if the total size of all the buffers is non-zero.
ByteBuffer buffer = nioBuffers[0];
int attemptedBytes = buffer.remaining();
//调用jdkchannel write
final int localWrittenBytes = ch.write(buffer);
if (localWrittenBytes <= 0) {
incompleteWrite(true);
return;
}
adjustMaxBytesPerGatheringWrite(attemptedBytes, localWrittenBytes, maxBytesPerGatheringWrite);
in.removeBytes(localWrittenBytes);
--writeSpinCount;
break;
}
default: {
// Zero length buffers are not added to nioBuffers by ChannelOutboundBuffer, so there is no need
// to check if the total size of all the buffers is non-zero.
// We limit the max amount to int above so cast is safe
long attemptedBytes = in.nioBufferSize();
final long localWrittenBytes = ch.write(nioBuffers, 0, nioBufferCnt);
if (localWrittenBytes <= 0) {
incompleteWrite(true);
return;
}
// Casting to int is safe because we limit the total amount of data in the nioBuffers to int above.
adjustMaxBytesPerGatheringWrite((int) attemptedBytes, (int) localWrittenBytes,
maxBytesPerGatheringWrite);
in.removeBytes(localWrittenBytes);
--writeSpinCount;
break;
}
}
} while (writeSpinCount > 0);
incompleteWrite(writeSpinCount < 0);
}
如果没有将buffer中的数据全部写完则，注册write事件到selector进行监听
protected final void incompleteWrite(boolean setOpWrite) {
// Did not write completely.
if (setOpWrite) {
setOpWrite();
} else {
// It is possible that we have set the write OP, woken up by NIO because the socket is writable, and then
// use our write quantum. In this case we no longer want to set the write OP because the socket is still
// writable (as far as we know). We will find out next time we attempt to write if the socket is writable
// and set the write OP if necessary.
clearOpWrite();
// Schedule flush again later so other tasks can be picked up in the meantime
eventLoop().execute(flushTask);
}
}
再次看NioEventLoop的是如何处理write事件的，
private void processSelectedKey(SelectionKey k, AbstractNioChannel ch) {
final AbstractNioChannel.NioUnsafe unsafe = ch.unsafe();
if (!k.isValid()) {
final EventLoop eventLoop;
try {
eventLoop = ch.eventLoop();
} catch (Throwable ignored) {
return;
}
if (eventLoop != this || eventLoop == null) {
return;
}
// close the channel if the key is not valid anymore
unsafe.close(unsafe.voidPromise());
return;
}
try {
int readyOps = k.readyOps();
// We first need to call finishConnect() before try to trigger a read(...) or write(...) as otherwise
// the NIO JDK channel implementation may throw a NotYetConnectedException.
if ((readyOps & SelectionKey.OP_CONNECT) != 0) {
// remove OP_CONNECT as otherwise Selector.select(..) will always return without blocking
// See https://github.com/netty/netty/issues/924
int ops = k.interestOps();
ops &= ~SelectionKey.OP_CONNECT;
k.interestOps(ops);
unsafe.finishConnect();
}
// Process OP_WRITE first as we may be able to write some queued buffers and so free memory.
if ((readyOps & SelectionKey.OP_WRITE) != 0) {
// Call forceFlush which will also take care of clear the OP_WRITE once there is nothing left to write
ch.unsafe().forceFlush();
}
// Also check for readOps of 0 to workaround possible JDK bug which may otherwise lead
// to a spin loop
if ((readyOps & (SelectionKey.OP_READ | SelectionKey.OP_ACCEPT)) != 0 || readyOps == 0) {
unsafe.read();
}
} catch (CancelledKeyException ignored) {
unsafe.close(unsafe.voidPromise());
}
}
可以看到直接调用了unsafe的flush0，flush0的操作见上面的代码，当然每次write都是调用outboundhandler的回调方法。
if ((readyOps & SelectionKey.OP_WRITE) != 0) {
// Call forceFlush which will also take care of clear the OP_WRITE once there is nothing left to write
ch.unsafe().forceFlush();
}
到此，读写事件处理也基本介绍完了。
总结
本篇文章主要梳理了netty基于Jdk NIo实现的工作流程，对其他的Nio实现其实是大同小异的，这里不进行分析了，有兴趣的童鞋可以自行参考上面的流程进行阅读源码。当然，对netty组件的细节实现还是有待更深入的考究，后续会对以下内容进行单独分析。
- ByteBuf的内部管理，内存分配和回收
- channalfuture、channelpromise
- channelpipeline和channelhandler的管理
- channelhandler、encoder、decoder处理粘包拆包
- 更多netty实战和相关实用的技术分析
以上是个人对netty这个优秀的网络nio框架工作原理的理解，欢迎一起交流！
参考
- java nio reactor模式https://blog.csdn.net/Houson_c/article/details/86114771
- netty实战 https://waylau.gitbooks.io/essential-netty-in-action/
- netty 权威指南
扫一扫
2165
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
