客户端的流程和服务端的大致相同,在服务端分析过的这里不再分析,看这篇文章前可以先看下服务端源码分析的文章。先大致看下客户端的代码。
public void start() throws InterruptedException {
EventLoopGroup group = new NioEventLoopGroup();/*线程组*/
try {
Bootstrap b = new Bootstrap();/*客户端启动必备*/
b.group(group)
.channel(NioSocketChannel.class)/*指明使用NIO进行网络通讯*/
.remoteAddress(new InetSocketAddress(host,port))/*配置远程服务器的地址*/
.handler(new EchoClientHandler());
ChannelFuture f = b.connect().sync();/*连接到远程节点,阻塞等待直到连接完成*/
/*阻塞,直到channel关闭*/
f.channel().closeFuture().sync();
} finally {
group.shutdownGracefully().sync();
}
}
调用Bootstrap的connect方法之前的代码,在之前服务端源码分析的文章中已经介绍了,这里不再重复,这里直接从调用connect方法开始分析。
1. connect
public ChannelFuture connect() {
//参数校验
validate();
SocketAddress remoteAddress = this.remoteAddress;
if (remoteAddress == null) {
throw new IllegalStateException("remoteAddress not set");
}
return doResolveAndConnect(remoteAddress, config.localAddress());
}
这里核心在doResolveAndConnect方法。
private ChannelFuture doResolveAndConnect(final SocketAddress remoteAddress, final SocketAddress localAddress) {
final ChannelFuture regFuture = initAndRegister();
final Channel channel = regFuture.channel();
if (regFuture.isDone()) {
if (!regFuture.isSuccess()) {
return regFuture;
}
return doResolveAndConnect0(channel, remoteAddress, localAddress, channel.newPromise());
} else {
// Registration future is almost always fulfilled already, but just in case it's not.
final PendingRegistrationPromise promise = new PendingRegistrationPromise(channel);
regFuture.addListener(new ChannelFutureListener() {
@Override
public void operationComplete(ChannelFuture future) throws Exception {
// Directly obtain the cause and do a null check so we only need one volatile read in case of a
// failure.
Throwable cause = future.cause();
if (cause != null) {
// Registration on the EventLoop failed so fail the ChannelPromise directly to not cause an
// IllegalStateException once we try to access the EventLoop of the Channel.
promise.setFailure(cause);
} else {
// Registration was successful, so set the correct executor to use.
// See https://github.com/netty/netty/issues/2586
promise.registered();
doResolveAndConnect0(channel, remoteAddress, localAddress, promise);
}
}
});
return promise;
}
}
1.1 initAndRegister
final ChannelFuture initAndRegister() {
Channel channel = null;
try {
//服务端不同的是,客户端这里创建的是NioSocketChannel
channel = channelFactory.newChannel();
init(channel);
} catch (Throwable t) {
if (channel != null) {
// channel can be null if newChannel crashed (eg SocketException("too many open files"))
channel.unsafe().closeForcibly();
// as the Channel is not registered yet we need to force the usage of the GlobalEventExecutor
return new DefaultChannelPromise(channel, GlobalEventExecutor.INSTANCE).setFailure(t);
}
// as the Channel is not registered yet we need to force the usage of the GlobalEventExecutor
return new DefaultChannelPromise(new FailedChannel(), GlobalEventExecutor.INSTANCE).setFailure(t);
}
ChannelFuture regFuture = config().group().register(channel);
if (regFuture.cause() != null) {
if (channel.isRegistered()) {
channel.close();
} else {
channel.unsafe().closeForcibly();
}
}
return regFuture;
}
这个方法和服务器启动的时候调用的是同一个方法,不过在细节上有所不同。
1.1.1 NioSocketChannel
服务端在这里创建的是NioServerSocketChannel,客户端这里创建的是NioSocketChannel。看下构造方法。
public NioSocketChannel() {
//DEFAULT_SELECTOR_PROVIDER是JDK原生的SelectorProvider
this(DEFAULT_SELECTOR_PROVIDER);
}
public NioSocketChannel(SelectorProvider provider) {
this(newSocket(provider));
}
newSocket利用SelectorProvider创建JDK的SocketChannel对象。
public NioSocketChannel(Channel parent, SocketChannel socket) {
//这里的parent是null
super(parent, socket);
config = new NioSocketChannelConfig(this, socket.socket());
}
super调用父类AbstractNioByteChannel的构造方法
protected AbstractNioByteChannel(Channel parent, SelectableChannel ch) {
super(parent, ch, SelectionKey.OP_READ);
}
继续调用父类AbstractNioChannel的构造方法
protected AbstractNioChannel(Channel parent, SelectableChannel ch, int readInterestOp) {
//父类构造方法创建了NioByteUnsafe对象和该channel的Pipeline
super(parent);
this.ch = ch;
//这里的感兴趣事件是read事件
this.readInterestOp = readInterestOp;
try {
//设置通道非阻塞
ch.configureBlocking(false);
} catch (IOException e) {
try {
ch.close();
} catch (IOException e2) {
if (logger.isWarnEnabled()) {
logger.warn(
"Failed to close a partially initialized socket.", e2);
}
}
throw new ChannelException("Failed to enter non-blocking mode.", e);
}
}
1.1.2 init
这里调用的是Bootstrap的init方法。
void init(Channel channel) throws Exception {
ChannelPipeline p = channel.pipeline();
//将配置的handler加入Pipeline
p.addLast(config.handler());
//设置属性
final Map<ChannelOption<?>, Object> options = options0();
synchronized (options) {
setChannelOptions(channel, options, logger);
}
final Map<AttributeKey<?>, Object> attrs = attrs0();
synchronized (attrs) {
for (Entry<AttributeKey<?>, Object> e: attrs.entrySet()) {
channel.attr((AttributeKey<Object>) e.getKey()).set(e.getValue());
}
}
}
1.1.3 register
register方法和服务端一样调用的是MultithreadEventLoopGroup的register方法。
public ChannelFuture register(Channel channel) {
return next().register(channel);
}
next().register调用的是SingleThreadEventLoop的register方法。
public ChannelFuture register(Channel channel) {
return register(new DefaultChannelPromise(channel, this));
}
public ChannelFuture register(final ChannelPromise promise) {
ObjectUtil.checkNotNull(promise, "promise");
promise.channel().unsafe().register(this, promise);
return promise;
}
这里将继续调用AbstractChannel内部类AbstractUnsafe的register方法。
public final void register(EventLoop eventLoop, final ChannelPromise promise) {
...//省略代码
AbstractChannel.this.eventLoop = eventLoop;
if (eventLoop.inEventLoop()) {
register0(promise);
} else {
try {
eventLoop.execute(new Runnable() {
@Override
public void run() {
register0(promise);
}
});
} catch (Throwable t) {
...//省略代码
}
}
}
这个方法之前讲过,就是分配一个eventLoop给channel,创建一个线程和eventLoop绑定,并且执行register0方法。
1.1.4 register0
private void register0(ChannelPromise promise) {
try {
...//省略代码
boolean firstRegistration = neverRegistered;
doRegister();
neverRegistered = false;
registered = true;
...//省略代码
pipeline.invokeHandlerAddedIfNeeded();
safeSetSuccess(promise);
pipeline.fireChannelRegistered();
...//省略代码
if (isActive()) {
if (firstRegistration) {
pipeline.fireChannelActive();
} else if (config().isAutoRead()) {
...//省略代码
beginRead();
}
}
} catch (Throwable t) {
...//省略代码
}
}
这个方法在之前也讲过,不分析了。回到doResolveAndConnect方法继续看接下来执行的doResolveAndConnect0方法。
1.2 doResolveAndConnect0
private ChannelFuture doResolveAndConnect0(final Channel channel, SocketAddress remoteAddress,
final SocketAddress localAddress, final ChannelPromise promise) {
try {
final EventLoop eventLoop = channel.eventLoop();
final AddressResolver<SocketAddress> resolver = this.resolver.getResolver(eventLoop);
if (!resolver.isSupported(remoteAddress) || resolver.isResolved(remoteAddress)) {
//执行这里的方法
doConnect(remoteAddress, localAddress, promise);
return promise;
}
final Future<SocketAddress> resolveFuture = resolver.resolve(remoteAddress);
if (resolveFuture.isDone()) {
final Throwable resolveFailureCause = resolveFuture.cause();
if (resolveFailureCause != null) {
// Failed to resolve immediately
channel.close();
promise.setFailure(resolveFailureCause);
} else {
// Succeeded to resolve immediately; cached? (or did a blocking lookup)
doConnect(resolveFuture.getNow(), localAddress, promise);
}
return promise;
}
// Wait until the name resolution is finished.
resolveFuture.addListener(new FutureListener<SocketAddress>() {
@Override
public void operationComplete(Future<SocketAddress> future) throws Exception {
if (future.cause() != null) {
channel.close();
promise.setFailure(future.cause());
} else {
doConnect(future.getNow(), localAddress, promise);
}
}
});
} catch (Throwable cause) {
promise.tryFailure(cause);
}
return promise;
}
这里将执行doConnect方法。
(1)doConnect
private static void doConnect(
final SocketAddress remoteAddress, final SocketAddress localAddress, final ChannelPromise connectPromise) {
// This method is invoked before channelRegistered() is triggered. Give user handlers a chance to set up
// the pipeline in its channelRegistered() implementation.
final Channel channel = connectPromise.channel();
channel.eventLoop().execute(new Runnable() {
@Override
public void run() {
if (localAddress == null) {
channel.connect(remoteAddress, connectPromise);
} else {
channel.connect(remoteAddress, localAddress, connectPromise);
}
connectPromise.addListener(ChannelFutureListener.CLOSE_ON_FAILURE);
}
});
}
这里的channel.connect将继续调用AbstractChannel的connect方法。
public ChannelFuture connect(SocketAddress remoteAddress, ChannelPromise promise) {
return pipeline.connect(remoteAddress, promise);
}
(2)connect
public final ChannelFuture connect(SocketAddress remoteAddress, ChannelPromise promise) {
return tail.connect(remoteAddress, promise);
}
connect是个出站事件,从TailContext开始出传播。我们现在这里有三个handler。HeadContext、EchoClientHandler(我们自定义的)、TailContext。
TailContext的connect方法是继承其父类AbstractChannelHandlerContext的方法。
public ChannelFuture connect(
final SocketAddress remoteAddress, final SocketAddress localAddress, final ChannelPromise promise) {
if (remoteAddress == null) {
throw new NullPointerException("remoteAddress");
}
if (isNotValidPromise(promise, false)) {
// cancelled
return promise;
}
//找到下一个出站事件去执行
final AbstractChannelHandlerContext next = findContextOutbound();
EventExecutor executor = next.executor();
if (executor.inEventLoop()) {
next.invokeConnect(remoteAddress, localAddress, promise);
} else {
safeExecute(executor, new Runnable() {
@Override
public void run() {
next.invokeConnect(remoteAddress, localAddress, promise);
}
}, promise, null);
}
return promise;
}
这里能处理出站事件的只有HeadContext。看下它的connect方法。
public void connect(
ChannelHandlerContext ctx,
SocketAddress remoteAddress, SocketAddress localAddress,
ChannelPromise promise) throws Exception {
unsafe.connect(remoteAddress, localAddress, promise);
}
这里将继续调用AbstractNioChannel内部类AbstractNioUnsafe的connect方法。
(3)AbstractNioUnsafe的connect
public final void connect(
final SocketAddress remoteAddress, final SocketAddress localAddress, final ChannelPromise promise) {
if (!promise.setUncancellable() || !ensureOpen(promise)) {
return;
}
try {
if (connectPromise != null) {
// Already a connect in process.
throw new ConnectionPendingException();
}
boolean wasActive = isActive();
if (doConnect(remoteAddress, localAddress)) {
fulfillConnectPromise(promise, wasActive);
} else {
...//省略代码}
} catch (Throwable t) {
promise.tryFailure(annotateConnectException(t, remoteAddress));
closeIfClosed();
}
}
看下doConnect方法。
(4)doConnect
protected boolean doConnect(SocketAddress remoteAddress, SocketAddress localAddress) throws Exception {
//如果本地地址不为空,绑定本地地址,我们这里是空的
if (localAddress != null) {
doBind0(localAddress);
}
boolean success = false;
try {
//调用JDK原生API连接远程主机
boolean connected = SocketUtils.connect(javaChannel(), remoteAddress);
if (!connected) {
//如果连接不成功,注册CONNECT事件
selectionKey().interestOps(SelectionKey.OP_CONNECT);
}
success = true;
return connected;
} finally {
if (!success) {
doClose();
}
}
}
到这里客户端启动完毕。
如果这里注册了CONNECT事件,后面连接成功,触发了CONNECT事件客户端怎么处理的。
2. connect事件
我们在上一篇文章中知道所有的处理事件都是由NioEventLoop的processSelectedKey方法处理。
private void processSelectedKey(SelectionKey k, AbstractNioChannel ch) {
final AbstractNioChannel.NioUnsafe unsafe = ch.unsafe();
if (!k.isValid()) {
final EventLoop eventLoop;
try {
eventLoop = ch.eventLoop();
} catch (Throwable ignored) {
...//省略代码
return;
}
...//省略代码
unsafe.close(unsafe.voidPromise());
return;
}
try {
int readyOps = k.readyOps();
...//省略代码
if ((readyOps & SelectionKey.OP_CONNECT) != 0) {
int ops = k.interestOps();
ops &= ~SelectionKey.OP_CONNECT;
//取消注册connect事件
k.interestOps(ops);
unsafe.finishConnect();
}
if ((readyOps & SelectionKey.OP_WRITE) != 0) {
// Call forceFlush which will also take care of clear the OP_WRITE once there is nothing left to write
ch.unsafe().forceFlush();
}
if ((readyOps & (SelectionKey.OP_READ | SelectionKey.OP_ACCEPT)) != 0 || readyOps == 0) {
unsafe.read();
}
} catch (CancelledKeyException ignored) {
unsafe.close(unsafe.voidPromise());
}
}
这里首先取消了注册的connect事件,然后调用了AbstractNioUnsafe的finishConnect方法。
(1)finishConnect
public final void finishConnect() {
assert eventLoop().inEventLoop();
try {
boolean wasActive = isActive();
doFinishConnect();
fulfillConnectPromise(connectPromise, wasActive);
} catch (Throwable t) {
fulfillConnectPromise(connectPromise, annotateConnectException(t, requestedRemoteAddress));
} finally {
// Check for null as the connectTimeoutFuture is only created if a connectTimeoutMillis > 0 is used
// See https://github.com/netty/netty/issues/1770
if (connectTimeoutFuture != null) {
connectTimeoutFuture.cancel(false);
}
connectPromise = null;
}
}
(2)doFinishConnect
protected void doFinishConnect() throws Exception {
if (!javaChannel().finishConnect()) {
throw new Error();
}
}
这里只是做了下检查
(3)fulfillConnectPromise
private void fulfillConnectPromise(ChannelPromise promise, boolean wasActive) {
if (promise == null) {
// Closed via cancellation and the promise has been notified already.
return;
}
boolean active = isActive();
// trySuccess() will return false if a user cancelled the connection attempt.
boolean promiseSet = promise.trySuccess();
if (!wasActive && active) {
pipeline().fireChannelActive();
}
// If a user cancelled the connection attempt, close the channel, which is followed by channelInactive().
if (!promiseSet) {
close(voidPromise());
}
}
这里最终调用了pipeline的fireChannelActive方法,这个方法在前一篇文章分析过,这里不分析了,最终它会在channel上注册read事件。
看到这里服务端和客户端的启动源码已经分析完了,accept、connect、read事件也分析过了,到这里是不是有点疑惑,还有个write事件在哪。
3. write
3.1 ChannelHandlerContext的writeAndFlush
我们在netty中写数据比较常用的是调用和handler绑定的ChannelHandlerContext的writeAndFlush方法。
这里调用的是AbstractChannelHandlerContext的writeAndFlush方法。
public ChannelFuture writeAndFlush(Object msg) {
return writeAndFlush(msg, newPromise());
}
public ChannelFuture writeAndFlush(Object msg, ChannelPromise promise) {
…//省略代码
write(msg, true, promise);
return promise;
}
(1)write
private void write(Object msg, boolean flush, ChannelPromise promise) {
//这里找能处理出站事件的handler,这里是HeadContext
AbstractChannelHandlerContext next = findContextOutbound();
final Object m = pipeline.touch(msg, next);
EventExecutor executor = next.executor();
if (executor.inEventLoop()) {
//判断是否需要flush
if (flush) {
next.invokeWriteAndFlush(m, promise);
} else {
next.invokeWrite(m, promise);
}
} else {
AbstractWriteTask task;
if (flush) {
task = WriteAndFlushTask.newInstance(next, m, promise);
} else {
task = WriteTask.newInstance(next, m, promise);
}
safeExecute(executor, task, promise, m);
}
}
这里先找下一个能处理出站事件的handler,这里能处理出站事件的只有HeadContext。
(2) invokeWriteAndFlush
这里会判断是否需要flush,如果需要调用HeadContext的invokeWriteAndFlush,否则调用invokeWrite。
private void invokeWriteAndFlush(Object msg, ChannelPromise promise) {
if (invokeHandler()) {
invokeWrite0(msg, promise);
invokeFlush0();
} else {
writeAndFlush(msg, promise);
}
}
private void invokeWrite(Object msg, ChannelPromise promise) {
if (invokeHandler()) {
invokeWrite0(msg, promise);
} else {
write(msg, promise);
}
}
这里可以看出来flush的方法只比不需要多了invokeFlush0方法。
(3)invokeWrite0
private void invokeWrite0(Object msg, ChannelPromise promise) {
try {
((ChannelOutboundHandler) handler()).write(this, msg, promise);
} catch (Throwable t) {
notifyOutboundHandlerException(t, promise);
}
这里调用的是HeadContext的write方法。
public void write(ChannelHandlerContext ctx, Object msg, ChannelPromise promise) throws Exception {
unsafe.write(msg, promise);
}
接着调用AbstractUnsafe的write方法。
(4)write
public final void write(Object msg, ChannelPromise promise) {
assertEventLoop();
ChannelOutboundBuffer outboundBuffer = this.outboundBuffer;
if (outboundBuffer == null) {
safeSetFailure(promise, WRITE_CLOSED_CHANNEL_EXCEPTION);
// release message now to prevent resource-leak
ReferenceCountUtil.release(msg);
return;
}
int size;
try {
msg = filterOutboundMessage(msg);
size = pipeline.estimatorHandle().size(msg);
if (size < 0) {
size = 0;
}
} catch (Throwable t) {
safeSetFailure(promise, t);
ReferenceCountUtil.release(msg);
return;
}
outboundBuffer.addMessage(msg, size, promise);
}
这个方法将数据写入了outboundBuffer。
(4)invokeFlush0
private void invokeFlush0() {
try {
((ChannelOutboundHandler) handler()).flush(this);
} catch (Throwable t) {
notifyHandlerException(t);
}
}
这里最终调用的还是AbstractUnsafe的flush方法。
(5)flush
public final void flush() {
assertEventLoop();
ChannelOutboundBuffer outboundBuffer = this.outboundBuffer;
if (outboundBuffer == null) {
return;
}
outboundBuffer.addFlush();
flush0();
}
这里调用了AbstractUnsafe的flush0方法。
protected void flush0() {
...//省略代码
inFlush0 = true;
...//省略代码
try {
doWrite(outboundBuffer);
} catch (Throwable t) {
...//省略代码
} else {
try {
shutdownOutput(voidPromise(), t);
} catch (Throwable t2) {
close(voidPromise(), t2, FLUSH0_CLOSED_CHANNEL_EXCEPTION, false);
}
}
} finally {
inFlush0 = false;
}
}
最终调用了NioSocketChannel的doWrite方法。
(6)doWrite
protected void doWrite(ChannelOutboundBuffer in) throws Exception {
SocketChannel ch = javaChannel();
//最多循环写入16次
int writeSpinCount = config().getWriteSpinCount();
do {
if (in.isEmpty()) {
// 如果没有要写的数据,清除write事件
clearOpWrite();
return;
}
// Ensure the pending writes are made of ByteBufs only.
int maxBytesPerGatheringWrite = ((NioSocketChannelConfig) config).getMaxBytesPerGatheringWrite();
ByteBuffer[] nioBuffers = in.nioBuffers(1024, maxBytesPerGatheringWrite);
int nioBufferCnt = in.nioBufferCount();
switch (nioBufferCnt) {
case 0:
// We have something else beside ByteBuffers to write so fallback to normal writes.
//这里是处理其他类型的数据写入
writeSpinCount -= doWrite0(in);
break;
case 1: {
//nioBuffers数量为1处理逻辑
ByteBuffer buffer = nioBuffers[0];
int attemptedBytes = buffer.remaining();
//写数据
final int localWrittenBytes = ch.write(buffer);
if (localWrittenBytes <= 0) {
incompleteWrite(true);
return;
}
adjustMaxBytesPerGatheringWrite(attemptedBytes, localWrittenBytes, maxBytesPerGatheringWrite);
in.removeBytes(localWrittenBytes);
--writeSpinCount;
break;
}
default: {
//nioBuffers数不为1和0处理逻辑
long attemptedBytes = in.nioBufferSize();
final long localWrittenBytes = ch.write(nioBuffers, 0, nioBufferCnt);
if (localWrittenBytes <= 0) {
//如果没有要写的数据,注册write事件
incompleteWrite(true);
return;
}
// Casting to int is safe because we limit the total amount of data in the nioBuffers to int above.
adjustMaxBytesPerGatheringWrite((int) attemptedBytes, (int) localWrittenBytes,
maxBytesPerGatheringWrite);
in.removeBytes(localWrittenBytes);
--writeSpinCount;
break;
}
}
} while (writeSpinCount > 0);
//writeSpinCount 大于0,代表在16次循环中数据已经写完,否则代表没写完
incompleteWrite(writeSpinCount < 0);
}
nioBufferCnt为0的处理逻辑,这个平时用的少不分析。nioBufferCnt为1和不为1的处理逻辑差不多,这里以1为例进行分析。
这个方法首先判断是否有要写入数据,没有的话调用clearOpWrite方法清除write事件。
protected final void clearOpWrite() {
final SelectionKey key = selectionKey();
if (!key.isValid()) {
return;
}
final int interestOps = key.interestOps();
if ((interestOps & SelectionKey.OP_WRITE) != 0) {
key.interestOps(interestOps & ~SelectionKey.OP_WRITE);
}
}
如果数据写完后,调用incompleteWrite方法重新注册write事件。
protected final void incompleteWrite(boolean setOpWrite) {
// Did not write completely.
if (setOpWrite) {
setOpWrite();
} else {
clearOpWrite();
// Schedule flush again later so other tasks can be picked up in the meantime
eventLoop().execute(flushTask);
}
}
protected final void setOpWrite() {
final SelectionKey key = selectionKey();
if (!key.isValid()) {
return;
}
final int interestOps = key.interestOps();
if ((interestOps & SelectionKey.OP_WRITE) == 0) {
key.interestOps(interestOps | SelectionKey.OP_WRITE);
}
}
如果有要写入的数据,则循环向buffer中写入数据,这里最多循环写入16次,最后调用incompleteWrite方法。
(7)incompleteWrite
protected final void incompleteWrite(boolean setOpWrite) {
// Did not write completely.
if (setOpWrite) {
setOpWrite();
} else {
clearOpWrite();
eventLoop().execute(flushTask);
}
}
如果在16次内已经把所有数据写完,则重新注册write事件,如果没有写完,先把write事件取消,再把flushTask加入线程等待队列。
我们前面分析过,线程的等待队列会在线程执行完所有在selector上注册的感兴趣事件后,会把等待队列中的任务执行完。
看下这个任务是什么
private final Runnable flushTask = new Runnable() {
@Override
public void run() {
// Calling flush0 directly to ensure we not try to flush messages that were added via write(...) in the
// meantime.
((AbstractNioUnsafe) unsafe()).flush0();
}
};
这个任务其实就是重新调用flush0写数据。写完后重新注册write事件。
3.1 write事件
最后我们看下如果注册了write事件,怎么处理的。
if ((readyOps & SelectionKey.OP_WRITE) != 0) {
// Call forceFlush which will also take care of clear the OP_WRITE once there is nothing left to write
ch.unsafe().forceFlush();
}
这里最后调用的是AbstractNioUnsafe的forceFlush方法。
public final void forceFlush() {
// directly call super.flush0() to force a flush now
super.flush0();
}
这里又回到调用AbstractUnsafe的flush0方法。上面已经分析过了。
到这里所有accept、connect、read、write事件已经分析完毕了。这里我们省略了不少过程,具体的可以在前一篇服务端的分析代码中去看看。