kafka源码(4)-基础数据结构之request&response&apis

目录

1.概述

2.解析request源码

2.1 RequestHeader

2.2 parseRequest

3.基础数据结构类

3.1Type类:

3.2 ArrayOf类:

3.3 Field类:

3.4 Schema类

3.5 Sturct类

3.6 ApiKeys

4.请求交互二进制数据格式

4.1 MetaDataRequest二进制格式

4.2 ProducerRequest二进制格式

4.3 FetchRequest二进制格式

.......

5.关系图:

---

1.概述

kafka启动时做很多初始化运行环境工作，具体请参考：kafka源码--broker的基础模块serversocket

其中SockeServer类启动时，首先初始化NIO网络环境、启动监听、创建主线程、工作线程池、设置参数等等。

kafka的所有逻辑处理和交互实际是交给KafkaApi类来处理的。

通过请求的类型，把具体的request路由到对应的handler处理。

2.解析request源码

2.1 RequestHeader

val header = RequestHeader.parse(receive.payload)

public class RequestHeader implements AbstractRequestResponse {

 public static RequestHeader parse(ByteBuffer buffer) {
        short apiKey = -1;
        try {
            apiKey = buffer.getShort();
            short apiVersion = buffer.getShort();
            short headerVersion = ApiKeys.forId(apiKey).requestHeaderVersion(apiVersion);
            // 设置position=0，mark=-1，为了后面重新读取
            buffer.rewind();
            return new RequestHeader(new RequestHeaderData(
                new ByteBufferAccessor(buffer), headerVersion), headerVersion);
        } catch (UnsupportedVersionException e) {
            throw new InvalidRequestException("Unknown API key " + apiKey, e);
        } catch (Throwable ex) {
            throw new InvalidRequestException("Error parsing request header. Our best guess of the apiKey is: " +
                    apiKey, ex);
        }
    }
}

public class RequestHeaderData implements ApiMessage {

    @Override
    public void read(Readable _readable, short _version) {
        // 读取2个字节数据设置到requestApiKey
        this.requestApiKey = _readable.readShort();
        // 读取2个字节数据设置到requestApiVersion
        this.requestApiVersion = _readable.readShort();
        // 读取4个字节数据设置到correlationId
        this.correlationId = _readable.readInt();
        if (_version >= 1) {
            int length;
            length = _readable.readShort();
            if (length < 0) {
                this.clientId = null;
            } else if (length > 0x7fff) {
                throw new RuntimeException("string field clientId had invalid length " + length);
            } else {
                this.clientId = _readable.readString(length);
            }
        } else {
            this.clientId = "";
        }
        this._unknownTaggedFields = null;
        if (_version >= 2) {
            int _numTaggedFields = _readable.readUnsignedVarint();
            for (int _i = 0; _i < _numTaggedFields; _i++) {
                int _tag = _readable.readUnsignedVarint();
                int _size = _readable.readUnsignedVarint();
                switch (_tag) {
                    default:
                        this._unknownTaggedFields = _readable.readUnknownTaggedField(this._unknownTaggedFields, _tag, _size);
                        break;
                }
            }
        }
    }

}

2.2 parseRequest

public class RequestContext implements AuthorizableRequestContext {

    public RequestAndSize parseRequest(ByteBuffer buffer) {
        if (isUnsupportedApiVersionsRequest()) {
            // Unsupported ApiVersion requests are treated as v0 requests and are not parsed
            ApiVersionsRequest apiVersionsRequest = new ApiVersionsRequest(new ApiVersionsRequestData(), (short) 0, header.apiVersion());
            return new RequestAndSize(apiVersionsRequest, 0);
        } else {
            ApiKeys apiKey = header.apiKey();
            try {
                short apiVersion = header.apiVersion();
                Struct struct = apiKey.parseRequest(apiVersion, buffer);
                AbstractRequest body = AbstractRequest.parseRequest(apiKey, apiVersion, struct);
                return new RequestAndSize(body, struct.sizeOf());
            } catch (Throwable ex) {
                throw new InvalidRequestException("Error getting request for apiKey: " + apiKey +
                        ", apiVersion: " + header.apiVersion() +
                        ", connectionId: " + connectionId +
                        ", listenerName: " + listenerName +
                        ", principal: " + principal, ex);
            }
        }
    }

}

public abstract class AbstractRequest implements AbstractRequestResponse {
    /**
     * Factory method for getting a requ