Kafka生产者介绍(七):创建发送请求

createProducerRequest
Sender.createProduceRequests()把待发送的消息封装成ClientRequest。无论一个Node对应多少个RecordBatch,或是发到多少个分区的,每个Node最多生成一个ClientRequest对象:
 

public class Sender implements Runnable {
    private List<ClientRequest> createProduceRequests(Map<Integer, List<RecordBatch>> collated, long now) {
        List<ClientRequest> requests = new ArrayList<ClientRequest>(collated.size());
        // 调用produceRequest方法,把发往同一个node的RecordBatch封装成一个ClientRequset对象。
        for (Map.Entry<Integer, List<RecordBatch>> entry : collated.entrySet())
            requests.add(produceRequest(now, entry.getKey(), acks, requestTimeout, entry.getValue()));
        return requests;
    }
    /**
     * Create a produce request from the given record batches
     */
    private ClientRequest produceRequest(long now, int destination, short acks, int timeout, List<RecordBatch> batches) {
        //produceRecordsByPartition,value是ByteBuffer
        Map<TopicPartition, ByteBuffer> produceRecordsByPartition = new HashMap<TopicPartition, ByteBuffer>(batches.size());
        // recordsByPartition,value是RecordBatch
        final Map<TopicPartition, RecordBatch> recordsByPartition = new HashMap<TopicPartition, RecordBatch>(batches.size());
        // 把batches里面的RecordBatch按照partiton进行分类,放入上列两个集合。
        for (RecordBatch batch : batches) {
            TopicPartition tp = batch.topicPartition;
            produceRecordsByPartition.put(tp, batch.records.buffer());
            recordsByPartition.put(tp, batch);
        // 创建ProduceRequest 和 RequestSend
        ProduceRequest request = new ProduceRequest(acks, timeout, produceRecordsByPartition);
        RequestSend send = new RequestSend(Integer.toString(destination),
                                           this.client.nextRequestHeader(ApiKeys.PRODUCE),
                                           request.toStruct());
        // RequestCompletionHandler为回调对象
        RequestCompletionHandler callback = new RequestCompletionHandler() {
            public void onComplete(ClientResponse response) {
                handleProduceResponse(response, recordsByPartition, time.milliseconds());
            }
        };
        // 创建ClientRequest,根据ack判断是否需要响应。
        return new ClientRequest(now, acks != 0, send, callback);
    }
}

1、将一个NodeId对应的RecordBatch集合,重新整理为produceRecordsByPartition和recordsByPartition两个集合
2、创建RequestSend和ProduceRequest
3、创建回调对象
4、创建并返回ClientRequest

后面会发送RequestSend对象,把ClientRequest放入InFlightRequest中缓存,当收到响应或者出现异常时候,通过缓存的ClientRequset调用其RequestCompletionHandler对象。
 

  • 1
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值