Netty通信 使用任务队列处理耗时操作
一、服务端与客户端的实现
二、任务队列的使用
此处仅展示NettyServerHandler的实现情况
1.用户自定义普通任务
1.代码
package com.example.netty.server.handler;
import io.netty.buffer.ByteBuf;
import io.netty.buffer.Unpooled;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.ChannelInboundHandlerAdapter;
import io.netty.util.CharsetUtil;
/**
* Description: Netty服务处理器
*
* @Author: zhx & moon hongxu_1234@163.com
* @Date: 2021-11-14 23:53
* @version: V1.0.0
*/
public class NettyServerHandler extends ChannelInboundHandlerAdapter {
/**
* 读取数据
* @param ctx 上下文对象 含有 管道 pipLine 通道 Channel 地址
* @param message 客户端发送的数据
*/
@Override
public void channelRead(ChannelHandlerContext ctx, Object message) throws Exception {
//将message转成一个ByteBuf
ByteBuf buf = (ByteBuf) message;
/**
* 在读取时插入一个耗时操作
*/
ctx.channel().eventLoop().execute(()->{
try {
Thread.sleep(10 * 1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
ctx.writeAndFlush(Unpooled.copiedBuffer("Hello Client,I take 10 seconds return",CharsetUtil.UTF_8));
});
}
/**
* 读取数据完毕
* @param ctx
* @throws Exception
*/
@Override
public void channelReadComplete(ChannelHandlerContext ctx) throws Exception {
//将数据写入到缓冲区并刷新
ctx.writeAndFlush(Unpooled.copiedBuffer("Hello Client",CharsetUtil.UTF_8));
}
/**
* 处理异常 一般为关闭通道
* @param ctx
* @param cause
* @throws Exception
*/
@Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
ctx.close();
}
}
2.结果
Connected to the target VM, address: '127.0.0.1:64033', transport: 'socket'
Java HotSpot(TM) 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
Client OK ...
Client :ChannelHandlerContext(NettyClientHandler#0, [id: 0x76372828, L:/127.0.0.1:64037 - R:/127.0.0.1:6668])
Server SendBack Message:Hello Client
Server Ip Address :/127.0.0.1:6668
Server SendBack Message:Hello Client,I take 10 seconds return
Server Ip Address :/127.0.0.1:6668
3.注意此处Execute是通过单一任务队列执行的,队内任务依次执行
4.实现类SingleThreadEventExecutor部分源码如下
@Override
public void execute(Runnable task) {
ObjectUtil.checkNotNull(task, "task");
execute(task, !(task instanceof LazyRunnable) && wakesUpForTask(task));
}
@Override
public void lazyExecute(Runnable task) {
execute(ObjectUtil.checkNotNull(task, "task"), false);
}
private void execute(Runnable task, boolean immediate) {
boolean inEventLoop = inEventLoop();
addTask(task);
if (!inEventLoop) {
startThread();
if (isShutdown()) {
boolean reject = false;
try {
if (removeTask(task)) {
reject = true;
}
} catch (UnsupportedOperationException e) {
// The task queue does not support removal so the best thing we can do is to just move on and
// hope we will be able to pick-up the task before its completely terminated.
// In worst case we will log on termination.
}
if (reject) {
reject();
}
}
}
if (!addTaskWakesUp && immediate) {
wakeup(inEventLoop);
}
}
5.验证
首先修改延时操作如下:
/**
* 在读取时插入一个耗时操作
*/
ctx.channel().eventLoop().execute(()->{
try {
Thread.sleep(10 * 1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
ctx.writeAndFlush(Unpooled.copiedBuffer("Hello Client,I take 10 seconds return",CharsetUtil.UTF_8));
System.out.println("First Task " + Thread.currentThread().getId() + " end :" + System.currentTimeMillis());
});
ctx.channel().eventLoop().execute(()->{
try {
Thread.sleep(20 * 1000);
System.out.println("Second Task " + Thread.currentThread().getId() + " end :" + System.currentTimeMillis());
} catch (InterruptedException e) {
e.printStackTrace();
}
ctx.writeAndFlush(Unpooled.copiedBuffer("Hello Client,I take 30 seconds return",CharsetUtil.UTF_8));
});
服务端日志:任务1和2结束相差大概20S
Connected to the target VM, address: '127.0.0.1:64223', transport: 'socket'
Java HotSpot(TM) 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
Server is Ready
Server Read Thread :nioEventLoopGroup-3-1
Server ctx=ChannelHandlerContext(NettyServerHandler#0, [id: 0x49c845d5, L:/127.0.0.1:6668 - R:/127.0.0.1:64229])
Client Send Message is:Hello Server
Client Ip Address is:/127.0.0.1:64229
First Task 21 end :1643211143862
Second Task 21 end :1643211163872
客户端结果日志:从最开始返回,计时 10S 和 30S 后分别有一次返回
Connected to the target VM, address: '127.0.0.1:64226', transport: 'socket'
Java HotSpot(TM) 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
Client OK ...
Client :ChannelHandlerContext(NettyClientHandler#0, [id: 0x424a1975, L:/127.0.0.1:64229 - R:/127.0.0.1:6668])
Server SendBack Message:Hello Client
Server Ip Address :/127.0.0.1:6668
Server SendBack Message:Hello Client,I take 10 seconds return
Server Ip Address :/127.0.0.1:6668
Server SendBack Message:Hello Client,I take 10 seconds return
Server Ip Address :/127.0.0.1:6668
Server SendBack Message:Hello Client,I take 30 seconds return
Server Ip Address :/127.0.0.1:6668
2.用户自定义定时任务
1.通过ScheduledFuture实现
/**
* 在读取时插入一个耗时操作
*/
System.out.println("Start to set common :" + System.currentTimeMillis());
ctx.channel().eventLoop().execute(()->{
try {
Thread.sleep(10 * 1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
ctx.writeAndFlush(Unpooled.copiedBuffer("Hello Client,I take 10 seconds return",CharsetUtil.UTF_8));
System.out.println("Common Task " + Thread.currentThread().getId() + " end " + System.currentTimeMillis());
});
/**
* 在读取时插入一个定时操作
*/
System.out.println("Start to set schedule :" + System.currentTimeMillis());
ctx.channel().eventLoop().schedule(()->{
System.out.println("Schedule Task " + Thread.currentThread().getId() + " start " + System.currentTimeMillis());
ctx.writeAndFlush(Unpooled.copiedBuffer("Hello Client,I take 5 seconds return",CharsetUtil.UTF_8));
},5, TimeUnit.SECONDS);
2.执行结果
可以看到,Scheduled任务和普通任务,还是同一线程执行的;且Scheduled设置为 5S 时,由于普通任务的阻塞,
Scheduled执行时,系统时间已经在创建任务时间的 10S 之后,所以Scheduled任务立即被执行
服务端日志
Connected to the target VM, address: '127.0.0.1:64302', transport: 'socket'
Java HotSpot(TM) 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
Server is Ready
Server Read Thread :nioEventLoopGroup-3-1
Server ctx=ChannelHandlerContext(NettyServerHandler#0, [id: 0xef412c6b, L:/127.0.0.1:6668 - R:/127.0.0.1:64309])
Client Send Message is:Hello Server
Client Ip Address is:/127.0.0.1:64309
Start to set common :1643212167383
Start to set schedule :1643212167385
Common Task 21 end 1643212177395
Schedule Task 21 start 1643212177397
客户端日志
Java HotSpot(TM) 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
Client OK ...
Client :ChannelHandlerContext(NettyClientHandler#0, [id: 0x20f963c1, L:/127.0.0.1:64309 - R:/127.0.0.1:6668])
Server SendBack Message:Hello Client
Server Ip Address :/127.0.0.1:6668
Server SendBack Message:Hello Client,I take 10 seconds return
Server Ip Address :/127.0.0.1:6668
Server SendBack Message:Hello Client,I take 5 seconds return
Server Ip Address :/127.0.0.1:6668
如果将Scheduled设为 15s 后执行,则通过日志可以看到,在普通任务执行后,又过了 5s 以后Scheduled才执行
Java HotSpot(TM) 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
Server is Ready
Server Read Thread :nioEventLoopGroup-3-1
Server ctx=ChannelHandlerContext(NettyServerHandler#0, [id: 0x3cf3ee82, L:/127.0.0.1:6668 - R:/127.0.0.1:64334])
Client Send Message is:Hello Server
Client Ip Address is:/127.0.0.1:64334
Start to set common :1643212459095
Start to set schedule :1643212459097
Common Task 21 end 1643212469108
Schedule Task 21 start 1643212474110
3.非Reactor线程,通过调用Channel获取任务队列,并添加任务
1.定义一个集合收集客户端连接分配的Channel
package com.example.netty.server;
import com.example.netty.server.handler.NettyServerHandler;
import io.netty.bootstrap.ServerBootstrap;
import io.netty.buffer.Unpooled;
import io.netty.channel.ChannelFuture;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.ChannelOption;
import io.netty.channel.EventLoopGroup;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioServerSocketChannel;
import io.netty.util.CharsetUtil;
import java.util.HashSet;
import java.util.Set;
/**
* Description: Netty服务端
*
* @Author: zhx & moon hongxu_1234@163.com
* @Date: 2021-11-14 23:31
* @version: V1.0.0
*/
public class NettyServer {
public static Set<SocketChannel> socketChannels = new HashSet<>(16);
public static void main(String[] args) throws InterruptedException {
//创建一个线程验证一下效果
new Thread(()->{
while (true){
try {
Thread.sleep(1000 * 1);
} catch (InterruptedException e) {
e.printStackTrace();
}
if (socketChannels.size() > 2){
break;
}
}
//开始广播
NettyServer.socketChannels.forEach(
socketChannel -> {
socketChannel.eventLoop().execute(()->{
socketChannel.pipeline().writeAndFlush(Unpooled.copiedBuffer("Hello Client , I am a broadcast", CharsetUtil.UTF_8));
});
}
);
}).start();
//创建BossGroup 和 WorkerGroup
/**
* 1.创建两个线程组
* 2.BossGroup 处理连接请求
* WorkerGroup 处理业务逻辑
* 3.两个都是无限循环
*/
EventLoopGroup bossGroup = new NioEventLoopGroup(1);
EventLoopGroup workerGroup = new NioEventLoopGroup();
try {
//创建服务端的启动对象 配置参数
ServerBootstrap bootstrap = new ServerBootstrap();
//使用链式编程进行设置
/**
* 1.设置两个线程组
* 2.使用 NioSocketChannel 作为服务器的通道实现
* 3.设置线程队列得到的连接数
* 4.设置保持活动连接状态
* 5.给我们的 WorkerGroup 的 EventLoop 对应的管道设置处理器
*/
bootstrap.group(bossGroup,workerGroup)
.channel(NioServerSocketChannel.class)
.option(ChannelOption.SO_BACKLOG,128)
.childOption(ChannelOption.SO_KEEPALIVE,true)
.childHandler(new ChannelInitializer<SocketChannel>() {
//给 PipLine 设置处理器
@Override
protected void initChannel(SocketChannel socketChannel) throws Exception {
System.out.println("Client Channel Hash " + socketChannel.hashCode());
socketChannels.add(socketChannel);
socketChannel.pipeline().addLast(new NettyServerHandler());
}
});
System.out.println("Server is Ready");
//绑定一个端口并且同步 生成了一个 ChannelFuture 对象 启动服务
ChannelFuture cf = bootstrap.bind(6668).sync();
//对关闭通道进行侦听
cf.channel().closeFuture().sync();
}finally {
bossGroup.shutdownGracefully();
workerGroup.shutdownGracefully();
}
}
}
2.使用CMD命令连接客户端并发送数据
telnet 127.0.0.1 6668
当第三个客户端请求连接时,断点查看集合内已有两个Channel
日志如下,每个Channel是独立的
Connected to the target VM, address: '127.0.0.1:64393', transport: 'socket'
Java HotSpot(TM) 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
Server is Ready
Client Channel Hash -1229417980
Server Read Thread :nioEventLoopGroup-3-1
Server ctx=ChannelHandlerContext(NettyServerHandler#0, [id: 0xc2540b4f, L:/127.0.0.1:6668 - R:/127.0.0.1:64397])
Client Send Message is:1
Client Ip Address is:/127.0.0.1:64397
Client Channel Hash -1652558443
Server Read Thread :nioEventLoopGroup-3-2
Server ctx=ChannelHandlerContext(NettyServerHandler#0, [id: 0x356c7bf1, L:/127.0.0.1:6668 - R:/127.0.0.1:64401])
Client Send Message is:2
Client Ip Address is:/127.0.0.1:64401
Client Channel Hash -1000753696
3.先启动程序的客户端,在通过CMD创建连个连接,查看上面定义的广播是否能触发
服务端日志
Connected to the target VM, address: '127.0.0.1:64575', transport: 'socket'
Java HotSpot(TM) 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
Server is Ready
Client Channel Hash -682928999
Server Read Thread :nioEventLoopGroup-3-1
Server ctx=ChannelHandlerContext(NettyServerHandler#0, [id: 0xd5a255f4, L:/127.0.0.1:6668 - R:/127.0.0.1:64581])
Client Send Message is:Hello Server
Client Ip Address is:/127.0.0.1:64581
Client Channel Hash 1558860967
Client Channel Hash 2037174976
客户端日志
Connected to the target VM, address: '127.0.0.1:64578', transport: 'socket'
Java HotSpot(TM) 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
Client OK ...
Client :ChannelHandlerContext(NettyClientHandler#0, [id: 0x764eb53b, L:/127.0.0.1:64581 - R:/127.0.0.1:6668])
Server SendBack Message:Hello Client
Server Ip Address :/127.0.0.1:6668
Server SendBack Message:Hello Client , I am a broadcast
Server Ip Address :/127.0.0.1:6668
广播成功触发