前言
netty的学习有以下几个难点:
- netty基于nio进行了较为复杂的封装,而很多童鞋对nio都不是很了解。
- netty中应用了Reactor模式,而Reactor模式本身有多种线程模型可以实现,netty用了较为复杂的那种。
线程按照其任务性质,可以分为工作线程和io线程,而netty线程主要就是io线程。我们知道,多线程程序有一些套路(设计模式),netty作为针对io线程的多线程程序,当然也有一定的套路。
netty对原生的nio作了封装,这决定了其实代码“套路”本质上跟原生nio是一致的,只是呈现出来的抽象不同。
为减少篇幅,本文涉及的所有代码忽略了次要部分及异常处理,所有代码可以在git@code.csdn.net:lqk654321/nio-demo.git
下载。
一般的nio代码如何写(单线程模型)
public class NIOServer {
public static void main(String[] args) throws IOException {
Selector selector = Selector.open();
ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
serverSocketChannel.configureBlocking(false);
serverSocketChannel.socket().bind(new InetSocketAddress(8080));
serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT);
while (true) {
selector.select(1000);
Set<SelectionKey> selectedKeys = selector.selectedKeys();
Iterator<SelectionKey> it = selectedKeys.iterator();
SelectionKey key = null;
while (it.hasNext()) {
key = it.next();
it.remove();
handleKey(key);
}
}
}
public static void handleKey(SelectionKey key) throws IOException {
if (key.isAcceptable()) {
// Accept the new connection
ServerSocketChannel ssc = (ServerSocketChannel) key.channel();
SocketChannel sc = ssc.accept();
sc.configureBlocking(false);
// Add the new connection to the selector
sc.register(key.selector(), SelectionKey.OP_READ | SelectionKey.OP_WRITE);
System.out.println("accept...");
} else if (key.isReadable()) {
SocketChannel sc = (SocketChannel) key.channel();
ByteBuffer readBuffer = ByteBuffer.allocate(1024);
// handle buffer
int count = sc.read(readBuffer);
if (count > 0) {
String receiveText = new String(readBuffer.array(), 0, count);
System.out.println("服务器端接受客户端数据--:" + receiveText);
}
}
}
}
代码中,这个while(true){监听并处理事件}
循环有个学名,叫eventloop。
在该示例中,所有工作放在一个线程中处理,很明显可靠性较低且性能不高。
- 从事件属性上讲,包括:accept事件、read/write事件。
- 从任务属性上讲,包括io任务,read/write数据的处理等
很明显我们要找到串行任务中的并行部分,将一个线程的事情拆分到多个线程中,拆分也将从“任务属性”和“事件属性”这两个维度来开展。
多线程模型
最容易想到的办法,当数据readable时,启动线程池,开启一个新的任务专门处理该数据,因此上节的handlerKey方法简化成了
public static void handleKey(SelectionKey key) throws IOException {
if (key.isAcceptable()) {
// Accept the new connection
ServerSocketChannel ssc = (ServerSocketChannel) key.channel();
SocketChannel sc = ssc.accept();
sc.configureBlocking(false);
// Add the new connection to the selector
sc.register(key.selector(), SelectionKey.OP_READ | SelectionKey.OP_WRITE);
System.out.println("accept...");
} else if (key.isReadable()) {
executor.execute(new Reader(key));
}
}
从事件属性出发,我们还可以再进一步,将readable和writable的判断也划到新的线程中去。
public static void handleKey(SelectionKey key) throws IOException {
if (key.isAcceptable()) {
// Accept the new connection
ServerSocketChannel ssc = (ServerSocketChannel) key.channel();
SocketChannel sc = ssc.accept();
sc.configureBlocking(false);
// Add the new connection to the selector
sc.register(key.selector(), SelectionKey.OP_READ | SelectionKey.OP_WRITE);
System.out.println("accept...");
} else {
executor.execute(new Worker(key));
}
}
主从线程模型
到目前为止,代码还没有完全并行化,因为acceptable事件和readable/writable事件的处理,也没什么关系。于是我们可以搞两个selector(当笔者看到可以两个selector监听不同事件的时候,一下子对netty豁然开朗了。笔者以前默认的意识里,一直以为selector只能有一个),一个负责监听acceptable事件,一个监听readable/writable事件的处理,分散在不同的线程中处理。
public class NIOServer {
private static ExecutorService boosExecutor = Executors.newFixedThreadPool(1);
private static ExecutorService workerExecutor = Executors.newFixedThreadPool(10);
private static Queue<SocketChannel> workerQueue = new ConcurrentLinkedQueue<SocketChannel>();
public static void main(String[] args) throws IOException {
/**
* boss只处理连接事件,worker只处理读写事件。
* 将两者分开的关键就是使用两个selector
*/
Selector bossSelector = Selector.open();
Selector workerSelector = Selector.open();
Boss boss = new Boss(bossSelector,workerQueue);
boss.bind();
boosExecutor.execute(boss);
workerExecutor.execute(new Worker(workerSelector,workerQueue));
}
}
boss线程实现
public class Boss implements Runnable {
public void bind() throws IOException {
ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
serverSocketChannel.configureBlocking(false);
serverSocketChannel.socket().bind(new InetSocketAddress(8080));
serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT);
}
public void run() {
while (true) {
selector.select();
Set<SelectionKey> selectedKeys = selector.selectedKeys();
Iterator<SelectionKey> it = selectedKeys.iterator();
SelectionKey key = null;
while (it.hasNext()) {
key = it.next();
it.remove();
handleKey(key);
}
}
}
private void handleKey(SelectionKey key) throws IOException {
if (key.isAcceptable()) {
// Accept the new connection
ServerSocketChannel ssc = (ServerSocketChannel) key.channel();
SocketChannel sc = ssc.accept();
System.out.println("boss connect...");
// 向woker队列中发送建立连接的SocketChannel
workerQueue.add(sc);
System.out.println("boss queue size " + workerQueue.size());
}
}
}
worker线程实现
public class Worker implements Runnable {
public void run() {
while (true) {
process(workerQueue);
process(selector);
Thread.sleep(1000);
}
}
public void process(Queue<SocketChannel> queue) throws IOException{
// 如果队列为空,会返回null,不会阻塞
SocketChannel sc = workerQueue.poll();
if(null != sc){
System.out.println("worker accept ...");
sc.configureBlocking(false);
sc.register(selector, SelectionKey.OP_READ | SelectionKey.OP_WRITE);
}
}
public void process(Selector selector) throws IOException{
//此处必须设置超时,因为最开始worker的selector没有绑定SocketChannel,所以“selector.select()会阻塞,并且再也无法恢复”
selector.select(1000);
// 处理读写事件
Set<SelectionKey> selectedKeys = selector.selectedKeys();
Iterator<SelectionKey> it = selectedKeys.iterator();
SelectionKey key = null;
while (it.hasNext()) {
key = it.next();
it.remove();
handleKey(key);
}
}
}
worker线程的主要工作就是:
- 查看下队列有没有新建立的连接
- 处理readable/writable事件
小结
netty中的boss和worker逻辑比这个复杂,但“意思是这个意思”,应该有助于读者理解netty源码。