mycat 1.6 源码分析-网络篇

相关资料

源码：https://github.com/MyCATApache/Mycat-Server
权威指南：http://www.mycat.org.cn/document/Mycat_V1.6.0.pdf
权威指南里面有入门篇，高级进阶篇，生产实践篇，开发篇。凝聚了各位开源大牛的心血，建议读者可结合权威指南和源码细读，应该会获益匪浅的。

启动

配置好server.xml，schema.xml和rule.xml，此处不做详细介绍，具体配置内容可参考权威指南（再次强调，权威指南很重要）。
然后在eclipse里面找到io.mycat.MycatStartup类，然后debug运行即可。

代码结构

此次分析的重点在io.mycat.net包下的类

NIOAcceptor

主要用途是打开ServerSocketChannel，设置相关属性，并向selector注册accept事件。

    public NIOAcceptor(String name, String bindIp,int port, 
            FrontendConnectionFactory factory, NIOReactorPool reactorPool)
            throws IOException {
        super.setName(name);
        this.port = port;
        this.selector = Selector.open();
        this.serverChannel = ServerSocketChannel.open();
        this.serverChannel.configureBlocking(false);
        /** 设置TCP属性 */
        serverChannel.setOption(StandardSocketOptions.SO_REUSEADDR, true);
        serverChannel.setOption(StandardSocketOptions.SO_RCVBUF, 1024 * 16 * 2);
        // backlog=100
        serverChannel.bind(new InetSocketAddress(bindIp, port), 100);
        this.serverChannel.register(selector, SelectionKey.OP_ACCEPT);
        this.factory = factory;
        this.reactorPool = reactorPool;
    }

开启NIOAcceptor线程接收客户端的连接请求。

    @Override
    public void run() {
        final Selector tSelector = this.selector;
        for (;;) {
            ++acceptCount;
            try {
                tSelector.select(1000L);
                Set<SelectionKey> keys = tSelector.selectedKeys();
                try {
                    for (SelectionKey key : keys) {
                        if (key.isValid() && key.isAcceptable()) {
                            accept();
                        } else {
                            key.cancel();
                        }
                    }
                } finally {
                    keys.clear();
                }
            } catch (Exception e) {
                LOGGER.warn(getName(), e);
            }
        }
    }

其中accept方法是接收客户端请求后的具体处理，ServerSocketChannel接收请求，然后利用FrontendConnectionFactory（包括ServerConnectionFactory前端连接工厂和ManagerConnectionFactory前端管理端连接工厂）进行创建FrontendConnection（ServerConnection+ManagerConnection），然后post到NIOReactorPool中的一个NIOReactor。

    private void accept() {
        SocketChannel channel = null;
        try {
            channel = serverChannel.accept();
            channel.configureBlocking(false);
            FrontendConnection c = factory.make(channel);
            c.setAccepted(true);
            c.setId(ID_GENERATOR.getId());
            NIOProcessor processor = (NIOProcessor) MycatServer.getInstance()
                    .nextProcessor();
            c.setProcessor(processor);

            NIOReactor reactor = reactorPool.getNextReactor();
            reactor.postRegister(c);

        } catch (Exception e) {
            LOGGER.warn(getName(), e);
            closeChannel(channel);
        }
    }

NIOReactorPool + NIOReactor

NIOReactorPool根据poolSize构建多个NIOReactor，并提供getNextReactor方法。

public class NIOReactorPool {
    private final NIOReactor[] reactors;
    private volatile int nextReactor;

    public NIOReactorPool(String name, int poolSize) throws IOException {
        reactors = new NIOReactor[poolSize];
        for (int i = 0; i < poolSize; i++) {
            NIOReactor reactor = new NIOReactor(name + "-" + i);
            reactors[i] = reactor;
            reactor.startup();
        }
    }

    public NIOReactor getNextReactor() {
//      if (++nextReactor == reactors.length) {
//          nextReactor = 0;
//      }
//      return reactors[nextReactor];

        int i = ++nextReactor;
        if (i >= reactors.length) {
            i=nextReactor = 0;
        }
        return reactors[i];
    }
}

NIOReactor类中的属性RW reactorR是主要作用是
1、把NIOAcceptor post 过来的连接向NIOReactor的Selector进行注册读事件
2、处理Selector轮询到的读写事件

        @Override
        public void run() {
            final Selector selector = this.selector;
            Set<SelectionKey> keys = null;
            for (;;) {
                ++reactCount;
                try {
                    selector.select(500L);
                    register(selector);
                    keys = selector.selectedKeys();
                    for (SelectionKey key : keys) {
                        AbstractConnection con = null;
                        try {
                            Object att = key.attachment();
                            if (att != null) {
                                con = (AbstractConnection) att;
                                if (key.isValid() && key.isReadable()) {
                                    try {
                                        //读事件处理
                                        con.asynRead();
                                    } catch (IOException e) {
                                        con.close("program err:" + e.toString());
                                        continue;
                                    } catch (Exception e) {
                                        LOGGER.warn("caught err:", e);
                                        con.close("program err:" + e.toString());
                                        continue;
                                    }
                                }
                                if (key.isValid() && key.isWritable()) {
                                    //写事件处理
                                    con.doNextWriteCheck();
                                }
                            } else {
                                key.cancel();
                            }
                        } catch (CancelledKeyException e) {
                            if (LOGGER.isDebugEnabled()) {
                                LOGGER.debug(con + " socket key canceled");
                            }
                        } catch (Exception e) {
                            LOGGER.warn(con + " " + e);
                        } catch (final Throwable e){
                            // Catch exceptions such as OOM and close connection if exists
                            //so that the reactor can keep running!
                            // @author Uncle-pan
                            // @since 2016-03-30
                            if(con != null){
                                con.close("Bad: "+e);
                            }
                            LOGGER.error("caught err: ", e);
                            continue;
                        }
                    }
                } catch (Exception e) {
                    LOGGER.warn(name, e);
                } catch (final Throwable e){
                    // Catch exceptions such as OOM so that the reactor can keep running!
                    // @author Uncle-pan
                    // @since 2016-03-30
                    LOGGER.error("caught err: ", e);
                } finally {
                    if (keys != null) {
                        keys.clear();
                    }

                }
            }
        }

NIOSocketWR

每个Connection中都有SocketWR对象，Connection的asynRead方法都是委托到NIOSocketWR 类处理（Connection中关于Socket读写都是委托此类处理），然后再调用AbstractConnection的onReadData(int got)方法进行处理，然后调用handle(byte[] data)方法，委托NioHandler.handle(byte[] data)方法处理（前端有FrontendAuthenticator，认证阶段通过后，Connection的NioHandler 设置为 FrontendCommandHandler处理命令阶段请求数据。）。

    @Override
    public void asynRead() throws IOException {
        ByteBuffer theBuffer = con.readBuffer;
        if (theBuffer == null) {

            theBuffer = con.processor.getBufferPool().allocate(con.processor.getBufferPool().getChunkSize());

            con.readBuffer = theBuffer;
        }

        int got = channel.read(theBuffer);

        con.onReadData(got);
    }

NIOConnector

后端的数据库连接创建后，调用postConnect(AbstractConnection c)提交NIOConnector的队列中。

    public void postConnect(AbstractConnection c) {
        connectQueue.offer(c);
        selector.wakeup();
    }

NIOConnector是一个线程，负责处理连接对象中注册到Selector的OP_CONNECT事件，启动后不断轮询，Selector.select(1000L) 找出就绪的OP_CONNECT事件，把之前post到NIOConnector的连接进行注册，然后处理就绪的OP_CONNECT事件，完成连接到后端Mysql。

    @Override
    public void run() {
        final Selector tSelector = this.selector;
        for (;;) {
            ++connectCount;
            try {
                tSelector.select(1000L);
                connect(tSelector);
                Set<SelectionKey> keys = tSelector.selectedKeys();
                try {
                    for (SelectionKey key : keys) {
                        Object att = key.attachment();
                        if (att != null && key.isValid() && key.isConnectable()) {
                            finishConnect(key, att);
                        } else {
                            key.cancel();
                        }
                    }
                } finally {
                    keys.clear();
                }
            } catch (Exception e) {
                LOGGER.warn(name, e);
            }
        }
    }

    private void connect(Selector selector) {
        AbstractConnection c = null;
        while ((c = connectQueue.poll()) != null) {
            try {
                SocketChannel channel = (SocketChannel) c.getChannel();
                channel.register(selector, SelectionKey.OP_CONNECT, c);
                channel.connect(new InetSocketAddress(c.host, c.port));

            } catch (Exception e) {
                LOGGER.error("error:",e);
                c.close(e.toString());
            }
        }
    }

    private void finishConnect(SelectionKey key, Object att) {
        BackendAIOConnection c = (BackendAIOConnection) att;
        try {
            if (finishConnect(c, (SocketChannel) c.channel)) {
                clearSelectionKey(key);
                c.setId(ID_GENERATOR.getId());
                NIOProcessor processor = MycatServer.getInstance()
                        .nextProcessor();
                c.setProcessor(processor);
                NIOReactor reactor = reactorPool.getNextReactor();
                reactor.postRegister(c);
                c.onConnectfinish();
            }
        } catch (Exception e) {
            clearSelectionKey(key);
            LOGGER.error("error:",e);
            c.close(e.toString());
            c.onConnectFailed(e);

        }
    }

    private boolean finishConnect(AbstractConnection c, SocketChannel channel)
            throws IOException {
        if (channel.isConnectionPending()) {
            channel.finishConnect();

            c.setLocalPort(channel.socket().getLocalPort());
            return true;
        } else {
            return false;
        }
    }

NIOProcessor

MycatServer启动时，创建多个NIOProcessor，然后ConnectionFactory创建连接的时候，分配一个NIOProcessor，NIOProcessor的主要作用是前后端连接的检查和流量的统计吧。

后面还需要继续了解的类

1. ClosableConnection NIOConnection AbstractConnection 及 其前后端连接的实现类
2. NIOHandler 及其实现类的用途