kafka协议-record解析

介绍

kafka的消息是以record方式的存储下来。

Record

Record是接口，DefaultRecord实现了Record接口。

DefaultRecord的存储结构

Record =>
  Length => varint    #  record总长度（不包括Length本身）
  Attributes => int8    # 属性
  TimestampDelta => varint    # timestamp的偏移量(相对于RecordBatch的baseTimestamp)
  OffsetDelta => varint    # offset的偏移量（相对于RecordBatch的baseOffset）
  KeyLen => varint    # key的长度
  Key => data    # key的数据
  ValueLen => varint    # value的长度
  Value => data    # value的数据
  NumHeaders => varint    # header的数量
  Headers => [Header]    # Header列表
  
Header =>
  HeaderKeyLen => varint    # key的长度
  HeaderKey => string    # key的数据
  HeaderValueLen => varint    # value的长度
  HeaderValue => data    # value的数据

上面数据类型有varint，这个类型是Protocol Buffers的类型。在存储数值比较小的时候，会节省空间。具体参考链接https://developers.google.com/protocol-buffers/docs/encoding

DefaultRecord类

DefaultRecord是上述数据结构的封装

public class DefaultRecord implements Record {

    // excluding key, value and headers: 5 bytes length + 10 bytes timestamp + 5 bytes offset + 1 byte attributes
    public static final int MAX_RECORD_OVERHEAD = 21;

    private static final int NULL_VARINT_SIZE_BYTES = ByteUtils.sizeOfVarint(-1);

    private final int sizeInBytes;
    private final byte attributes;
    private final long offset;
    private final long timestamp;
    private final int sequence;
    private final ByteBuffer key;
    private final ByteBuffer value;
    private final Header[] headers;

    private DefaultRecord(int sizeInBytes,
                          byte attributes,
                          long offset,
                          long timestamp,
                          int sequence,
                          ByteBuffer key,
                          ByteBuffer value,
                          Header[] headers) {
        this.sizeInBytes = sizeInBytes;
        this.attributes = attributes;
        this.offset = offset;
        this.timestamp = timestamp;
        this.sequence = sequence;
        this.key = key;
        this.value = value;
        this.headers = headers;
    }

    @Override
    public long offset