MQ!Rabbit-client FrameHandler

MQ!Rabbit-client FrameHandler

上一边写了关于 RabbitMq-clien 建立链接的源码

说到创建链接的神龙需要3颗龙珠分别是 ConnectionParams FrameHandler MetricsCollector

• MetricsCollector是官方开放的一个扩展点，开发人员实现这个接口并且传递即可实现自己想做的事情。以后有需要我们在看这个官方的相关实现类。

• ConnectionParams 就是一个实体类存放了参数信息

• FrameHandler 这个是本次学习的部分。

什么是Frame

这里需要理解下AMQP协议(又臭又长，不想看可以直接看这一节下面的小结🧐)

参考地址：

官网（纯英文）：https://www.amqp.org/specification/0-9-1/amqp-org-download

http://www.blogjava.net/qbna350816/archive/2016/08/12/431554.html

英文好的小伙伴建议也看官网下载的纯英文文档（有需要的不想自己下的可以评论留下邮箱），另外一篇是网上找的一篇，解释的也挺不错的。

官方文档下载下来解压后可以看到4个文件：

• amqp-0-9-1.pdf
• amqp0-9-1.stripped.xml
• amqp-xml-doc0-9-1.pdf
• amqp0-9-1.xml

看英文版文档（amqp-0-9-1.pdf）中【1.2.4 The Advanced Message Queuing Protocol (AMQP) 】

The AMQP protocol is a binary protocol with modern features: it is multi-channel, negotiated, asynchronous, secure, portable, neutral, and efficient.  AMQP is usefully split into two layers: 

+-----------    Functional Layer -------------------------------+
+    Basic  Transactions  Exchanges  Message queues             +
+-----------    Transport Layer --------------------------------+
+    Framing  Content  Data representation                      +
+    Error handling  Heart-beating   Channels                   +

前面那几句英文就是说自己产品多么多么好，具有什么多管道、协商、异步、安全、便携（轻量？）、高效的协议（😲这个可以学一下，以后万一自己做出来的产品也可以套用下这写词手动滑稽，当然AMQP的确有这些特性）。

接下来这句话就关键：AMQP is usefully split into two layers（AMQP分为2部分）
1、 Functional Layer ： 功能层

2、Transport Layer ： 传输层

The functional layer defines a set of commands (grouped into logical classes of functionality) that do useful work on behalf of the application. 

The transport layer that carries these methods from application to server, and back, and which handles channel multiplexing, framing, content encoding, heart-beating, data representation, and error handling

功能层定义了一组命令(分组为功能的逻辑类)，它们用来保证应用程序正常工作。

传输层将这些方法传递给服务器并返回。同时还会用来处理多路复用、帧、字符编码、心跳、数据表示及异常错误的处理

这里我们可以看到传输层(Transport Layer)中有处理 帧（framing）。

那么接下里就寻找和frame相关的地方

2.3.4 Delimiting Frames  ~  2.3.5.3 Heartbeat Frames 
--------------------------------------------------------------------------------------
2.3.4 Delimiting Frames 
TCP/IP is a stream protocol, i.e. there is no in-built mechanism for delimiting frames.  Existing protocols solve this in several different ways: 
 Sending a single frame per connection. This is simple but slow. 
 Adding frame delimiters to the stream. This is simple but slow to parse. 
 Counting the size of frames and sending the size in front of each frame. This is simple and fast, and our choice.

帧的分离
TCP/IP是一个流协议，即没有分隔帧的内建机制. 现有协议可以几种不同的方式解决这个问题
1、每个连接中只发送单个帧，简单但缓慢 
2、在流中添加帧定界符.简单，但解析较慢.
3、AMQP的选择的是计算帧的大小， 并在每个帧的前面(帧头部)发送大小。这是简单和快速


2.3.5 Frame Details 
All frames consist of a header (7 octets), a payload of arbitrary size, and a 'frame-end' octet that detects malformed frames

 0      1         3             7                  size+7 size+8  
 +------+---------+-------------+  +------------+  +-----------+  
 | type | channel |     size    |  |  payload   |  | frame-end |  
 +------+---------+-------------+  +------------+  +-----------+   
 octet   short         long         size octets       octet 
 
To read a frame, we: 
1. Read the header and check the frame type and channel. 
2. Depending on the frame type, we read the payload and process it. 
3. Read the frame end octet. In realistic implementations where performance is a concern, we would use “read-ahead buffering” or “gathering reads” to avoid doing three separate system calls to read a frame.


所有的帧都由一个头(header,7个字节),任意大小的内容字节(payload),和一个检测错误帧的frame-end结束

读取帧：
1. 读取header,检查帧类型(frame type)和通道(channel).
2. 根据帧类型，我们读取负载并进行处理.
3. 读取帧结束字节.在实际实现中，如果性能很关键的话，我们应该使用读前缓冲（read-ahead buffering）”或“收集读取（gathering reads）”，以避免为了读一个帧而做三次独立的系统调用。

后面我就不在放英文原文了，直接写中文理解

帧的类型分为
方法帧、内容帧、心跳帧。具体处理方式我这里就不写了，有兴趣的可以看官方文档

小结：

• frame可以理解为帧，即AMQP中通讯的基本单位。

• frame结构为 frame-header、 payload、 frame-end

  • frame-header: type（帧类型） + channel（通道） + size（内容大小）共7个字节
  • payload: 内容负载或者内容主体
  • frame-end：帧结束的校验字节

• frame的类型分为 方法帧、内容帧<内容头帧(conent header frame)和零个或多个内容体帧(content body frame)>、 心跳帧。

• frame处理的大概流程

  • 读取header,检查帧类型(frame type)和通道(channel)
  • 根据帧的类型(type)和payload的长度(size)来读取指定大小的payload进行处理
  • 读取最后一个字节(结束帧)。

  不同类型的帧处理方式不相同。不同帧的具体细节我不写了（🙄，其实我也没看）

frame源码

com.rabbitmq.client.impl.Frame

public class Frame {
    /** Frame type code */
    public final int type;

    /** Frame channel number, 0-65535 */
    public final int channel;

    /** Frame payload bytes (for inbound frames) */
    private final byte[] payload;

    /** Frame payload (for outbound frames) */
    private final ByteArrayOutputStream accumulator;

    private static final int NON_BODY_SIZE = 1 /* type */ + 2 /* channel */ + 4 /* payload size */ + 1 /* end character */;
    
    ...
        
        
    /**
     * Protected API - Factory method to instantiate a Frame by reading an
     * AMQP-wire-protocol frame from the given input stream.
     *
     * @return a new Frame if we read a frame successfully, otherwise null
     */
    public static Frame readFrom(DataInputStream is) throws IOException {
        int type;
        int channel;

        try {
            // 读取流中的第一个字节type
            type = is.readUnsignedByte();
        } catch (SocketTimeoutException ste) {
            // System.err.println("Timed out waiting for a frame.");
            return null; // failed
        }

        if (type == 'A') {
            /*
             * Probably an AMQP.... header indicating a version
             * mismatch.
             */
            /*
             * Otherwise meaningless, so try to read the version,
             * and throw an exception, whether we read the version
             * okay or not.
             */
            // 检查协议版本
            protocolVersionMismatch(is);
        }
		// 又读了一个字节 channel id
        channel = is.readUnsignedShort();
        // 读取负载size大小
        int payloadSize = is.readInt();
        byte[] payload = new byte[payloadSize];
        // 读取payload
        is.readFully(payload);

        // 读取结束位
        int frameEndMarker = is.readUnsignedByte();
        if (frameEndMarker != AMQP.FRAME_END) {
            throw new MalformedFrameException("Bad frame end marker: " + frameEndMarker);
        }

        return new Frame(type, channel, payload);
    }    
}

FrameHandler

接下来我们的重头戏看下FrameHandler

实现类

• SocketFrameHandler: IO方式
• SocketChannelFrameHandler NIO 方式

上一篇我们分析的是IO方式，看下源码中的龙珠2号生成器是什么

// 七龙珠2号生产厂家 FrameHandlerFactory
FrameHandlerFactory fhFactory = createFrameHandlerFactory();

• FrameHandlerFactory

这个就是IO版本的，同样也存在一个NIO版本的生产厂家

• SocketChannelFrameHandlerFactory

我们先来看工厂的源码（因为这个地方我们上一篇已经分析到时通过工厂创建frameHandler了。）

工厂：FrameHandlerFactory & SocketChannelFrameHandlerFactory

• SocketFrameHandlerFactory

public class SocketFrameHandlerFactory extends AbstractFrameHandlerFactory {

    private final SocketFactory factory;
    //用于关闭socket的线程池
    private final ExecutorService shutdownExecutor;

    public SocketFrameHandlerFactory(int connectionTimeout, SocketFactory factory, SocketConfigurator configurator, boolean ssl) {
        this(connectionTimeout, factory, configurator, ssl, null);
    }

    public SocketFrameHandlerFactory(int connectionTimeout, SocketFactory factory, SocketConfigurator configurator, boolean ssl, ExecutorService shutdownExecutor) {
        super(connectionTimeout, configurator, ssl);
        this.factory = factory;
        this.shutdownExecutor = shutdownExecutor;
    }

    public FrameHandler create(Address addr) throws IOException {
        String hostName = addr.getHost();
        int portNumber = ConnectionFactory.portOrDefault(addr.getPort(), ssl);
        Socket socket = null;
        try {
            // 通过socketFactory创建Socket
            socket = factory.createSocket();
            // 设置socket选项
            configurator.configure(socket);
            // 开启连接
            socket.connect(new InetSocketAddress(hostName, portNumber),
                    connectionTimeout);
            return create(socket);
        } catch (IOException ioe) {
            quietTrySocketClose(socket);
            throw ioe;
        }
    }

    public FrameHandler create(Socket sock) throws IOException
    {
        return new SocketFrameHandler(sock, this.shutdownExecutor);
    }

    private static void quietTrySocketClose(Socket socket) {
        if (socket != null)
            try { socket.close(); } catch (Exception _e) {/*ignore exceptions*/}
    }
}

• SocketChannelFrameHandlerFactory （……没什么可写的😂）

public class SocketChannelFrameHandlerFactory extends AbstractFrameHandlerFactory {

    final NioParams nioParams;

    private final SslContextFactory sslContextFactory;

    private final Lock stateLock = new ReentrantLock();

    private final AtomicLong globalConnectionCount = new AtomicLong();

    private final List<NioLoopContext> nioLoopContexts;

    public SocketChannelFrameHandlerFactory(int connectionTimeout, NioParams nioParams, boolean ssl, SslContextFactory sslContextFactory)
        throws IOException {
        super(connectionTimeout, null, ssl);
        this.nioParams = new NioParams(nioParams);
        this.sslContextFactory = sslContextFactory;
        this.nioLoopContexts = new ArrayList<NioLoopContext>(this.nioParams.getNbIoThreads());
        for (int i = 0; i < this.nioParams.getNbIoThreads(); i++) {
            this.nioLoopContexts.add(new NioLoopContext(this, this.nioParams));
        }
    }

    @Override
    public FrameHandler create(Address addr, String connectionName) throws IOException {
        int portNumber = ConnectionFactory.portOrDefault(addr.getPort(), ssl);

        SSLEngine sslEngine = null;
        SocketChannel channel = null;

        try {
            if (ssl) {
                SSLContext sslContext = sslContextFactory.create(connectionName);
                sslEngine = sslContext.createSSLEngine(addr.getHost(), portNumber);
                sslEngine.setUseClientMode(true);
                if (nioParams.getSslEngineConfigurator() != null) {
                    nioParams.getSslEngineConfigurator().configure(sslEngine);
                }
            }

            SocketAddress address = new InetSocketAddress(addr.getHost(), portNumber);
            channel = SocketChannel.open();
            channel.configureBlocking(true);
            if(nioParams.getSocketChannelConfigurator() != null) {
                nioParams.getSocketChannelConfigurator().configure(channel);
            }

            channel.connect(address);

            if (ssl) {
                sslEngine.beginHandshake();
                boolean handshake = SslEngineHelper.doHandshake(channel, sslEngine);
                if (!handshake) {
                    throw new SSLException("TLS handshake failed");
                }
            }

            channel.configureBlocking(false);

            // lock
            stateLock.lock();
            NioLoopContext nioLoopContext = null;
            try {
                long modulo = globalConnectionCount.getAndIncrement() % nioParams.getNbIoThreads();
                nioLoopContext = nioLoopContexts.get((int) modulo);
                nioLoopContext.initStateIfNecessary();
                SocketChannelFrameHandlerState state = new SocketChannelFrameHandlerState(
                    channel,
                    nioLoopContext,
                    nioParams,
                    sslEngine
                );
                state.startReading();
                SocketChannelFrameHandler frameHandler = new SocketChannelFrameHandler(state);
                return frameHandler;
            } finally {
                stateLock.unlock();
            }


        } catch(IOException e) {
            try {
                if(sslEngine != null && channel != null) {
                    SslEngineHelper.close(channel, sslEngine);
                }
                channel.close();
            } catch(IOException closingException) {
                // ignore
            }
            throw e;
        }

    }

    void lock() {
        stateLock.lock();
    }

    void unlock() {
        stateLock.unlock();
    }
}

操作类：SocketFrameHandler & SocketChannelFrameHandler

• SocketFrameHandler

属性：持有DataOutputStream DataInputStream Socket 等实例，用来处理frame

/** The underlying socket */
private final Socket _socket;

/**
  * Optional {@link ExecutorService} for final flush.
  */
private final ExecutorService _shutdownExecutor;

/** Socket's inputstream - data from the broker - synchronized on */
private final DataInputStream _inputStream;

/** Socket's outputstream - data to the broker - synchronized on */
private final DataOutputStream _outputStream;

/** Time to linger before closing the socket forcefully. */
public static final int SOCKET_CLOSING_TIMEOUT = 1;

close（）方法：

@Override
public void close() {
    try { _socket.setSoLinger(true, SOCKET_CLOSING_TIMEOUT); } catch (Exception _e) {}
    // async flush if possible
    // see https://github.com/rabbitmq/rabbitmq-java-client/issues/194
    Callable<Void> flushCallable = new Callable<Void>() {
        @Override
        public Void call() throws Exception {
            flush();
            return null;
        }
    };
    Future<Void> flushTask = null;
    try {
        if(this._shutdownExecutor == null) {
            flushCallable.call();
        } else {
            flushTask = this._shutdownExecutor.submit(flushCallable);
            flushTask.get(SOCKET_CLOSING_TIMEOUT, TimeUnit.SECONDS);
        }
    } catch(Exception e) {
        if(flushTask != null) {
            flushTask.cancel(true);
        }
    }
    try { _socket.close(); } catch (Exception _e) {}
}

其他方法（略）

• SocketChannelFrameHandler

查看源码发现这个类中唯一的属性

private final SocketChannelFrameHandlerState state;

而其他方法其实都时调用这个state的api。这里可以将 SocketChannelFrameHandler 理解为 SocketChannelFrameHandlerState 的代理。那么下面就继续看下SocketChannelFrameHandlerState

• SocketChannelFrameHandlerState

属性

private static final int SOCKET_CLOSING_TIMEOUT = 1;
private final SocketChannel channel;
private final BlockingQueue<WriteRequest> writeQueue;
private volatile AMQConnection connection;
private long lastActivity;
// 写 selector
private final SelectorHolder writeSelectorState;
// 读 selector
private final SelectorHolder readSelectorState;

private final int writeEnqueuingTimeoutInMs;
final boolean ssl;
final SSLEngine sslEngine;
final ByteBuffer plainOut;
final ByteBuffer plainIn;
final ByteBuffer cipherOut;
final ByteBuffer cipherIn;
final DataOutputStream outputStream;
final FrameBuilder frameBuilder;

构造

public SocketChannelFrameHandlerState(SocketChannel channel, NioLoopContext nioLoopsState, NioParams nioParams, SSLEngine sslEngine) {
        this.channel = channel;
        this.readSelectorState = nioLoopsState.readSelectorState;
        this.writeSelectorState = nioLoopsState.writeSelectorState;
        this.writeQueue = new ArrayBlockingQueue<WriteRequest>(nioParams.getWriteQueueCapacity(), true);
        this.writeEnqueuingTimeoutInMs = nioParams.getWriteEnqueuingTimeoutInMs();
        this.sslEngine = sslEngine;
        if(this.sslEngine == null) {
            this.ssl = false;
            this.plainOut = nioLoopsState.writeBuffer;
            this.cipherOut = null;
            this.plainIn = nioLoopsState.readBuffer;
            this.cipherIn = null;

            this.outputStream = new DataOutputStream(
                new ByteBufferOutputStream(channel, plainOut)
            );

            this.frameBuilder = new FrameBuilder(channel, plainIn);

        } else {
            this.ssl = true;
            this.plainOut = ByteBuffer.allocate(sslEngine.getSession().getApplicationBufferSize());
            this.cipherOut = ByteBuffer.allocate(sslEngine.getSession().getPacketBufferSize());
            this.plainIn = ByteBuffer.allocate(sslEngine.getSession().getApplicationBufferSize());
            this.cipherIn = ByteBuffer.allocate(sslEngine.getSession().getPacketBufferSize());

            this.outputStream = new DataOutputStream(
                new SslEngineByteBufferOutputStream(sslEngine, plainOut, cipherOut, channel)
            );
            this.frameBuilder = new SslEngineFrameBuilder(sslEngine, plainIn, cipherIn, channel);
        }

    }

发送

private void sendWriteRequest(WriteRequest writeRequest) throws IOException {
        try {
            boolean offered = this.writeQueue.offer(writeRequest, writeEnqueuingTimeoutInMs, TimeUnit.MILLISECONDS);
            if(offered) {
                this.writeSelectorState.registerFrameHandlerState(this, SelectionKey.OP_WRITE);
                this.readSelectorState.selector.wakeup();
            } else {
                throw new IOException("Frame enqueuing failed");
            }
        } catch (InterruptedException e) {
            LOGGER.warn("Thread interrupted during enqueuing frame in write queue");
        }
}