rocketmq 之namesrv（十）mqclient admin请求处理根据时间戳查找消息的物理偏移量

最新推荐文章于 2023-07-05 04:37:45 发布

游语

最新推荐文章于 2023-07-05 04:37:45 发布

阅读量496

点赞数 1

分类专栏： rocketmq

本文链接：https://blog.csdn.net/qq_26400953/article/details/103187195

版权

rocketmq 专栏收录该内容

48 篇文章 4 订阅

订阅专栏

根据时间戳查找消息的物理偏移量 AdminBrokerProcessor#processRequest#searchOffsetByTimestamp

//    查找存储时间戳为指定值的消息的物理偏移量
    private RemotingCommand searchOffsetByTimestamp(ChannelHandlerContext ctx,
        RemotingCommand request) throws RemotingCommandException {
        final RemotingCommand response = RemotingCommand.createResponseCommand(SearchOffsetResponseHeader.class);
        final SearchOffsetResponseHeader responseHeader = (SearchOffsetResponseHeader) response.readCustomHeader();
        final SearchOffsetRequestHeader requestHeader =
            (SearchOffsetRequestHeader) request.decodeCommandCustomHeader(SearchOffsetRequestHeader.class);

        long offset = this.brokerController.getMessageStore().getOffsetInQueueByTime(requestHeader.getTopic(), requestHeader.getQueueId(),
            requestHeader.getTimestamp());

        responseHeader.setOffset(offset);

        response.setCode(ResponseCode.SUCCESS);
        response.setRemark(null);
        return response;
    }

DefaultMessageStore#getOffsetInQueueByTime

public long getOffsetInQueueByTime(String topic, int queueId, long timestamp) {
        //获取消费者队列
        ConsumeQueue logic = this.findConsumeQueue(topic, queueId);
        if (logic != null) {
            return logic.getOffsetInQueueByTime(timestamp);
        }

        return 0;
    }

获取消费者队列DefaultMessageStore#findConsumeQueue

public ConsumeQueue findConsumeQueue(String topic, int queueId) {
        ConcurrentMap<Integer, ConsumeQueue> map = consumeQueueTable.get(topic);
        //如果根据topic获取到的队列map为空
        if (null == map) {
            ConcurrentMap<Integer, ConsumeQueue> newMap = new ConcurrentHashMap<Integer, ConsumeQueue>(128);
            //如果topic存在 队列map为空则创建一个map并关联
            ConcurrentMap<Integer, ConsumeQueue> oldMap = consumeQueueTable.putIfAbsent(topic, newMap);
            if (oldMap != null) {
                map = oldMap;
            } else {
                map = newMap;
            }
        }

//        如果消费者的消费队列为空创建一个放进去
        ConsumeQueue logic = map.get(queueId);
        if (null == logic) {
            ConsumeQueue newLogic = new ConsumeQueue(
                topic,
                queueId,
                StorePathConfigHelper.getStorePathConsumeQueue(this.messageStoreConfig.getStorePathRootDir()),
                this.getMessageStoreConfig().getMapedFileSizeConsumeQueue(),
                this);
            ConsumeQueue oldLogic = map.putIfAbsent(queueId, newLogic);
            if (oldLogic != null) {
                logic = oldLogic;
            } else {
                logic = newLogic;
            }
        }

        return logic;
    }

MessageStoreConfig#getMapedFileSizeConsumeQueue

this.getMessageStoreConfig().getMapedFileSizeConsumeQueue()

public int getMapedFileSizeConsumeQueue() {

       int factor = (int) Math.ceil(this.mapedFileSizeConsumeQueue / (ConsumeQueue.CQ_STORE_UNIT_SIZE * 1.0));
       return (int) (factor * ConsumeQueue.CQ_STORE_UNIT_SIZE);
   }

这一个操作是强制把消费队列文件大小改为20的倍数，原因是消费队列文件最小单元为20
ConsumeQueue.CQ_STORE_UNIT_SIZE:
8 字节的 commitlog 物理偏移量、4 字节的消息长度、8 字节 tag hashcode

ConsumeQueue#getOffsetInQueueByTime

logic.getOffsetInQueueByTime(timestamp)

public long getOffsetInQueueByTime(final long timestamp) {
       //获取指定时间段的消息存储文件
       MappedFile mappedFile = this.mappedFileQueue.getMappedFileByTime(timestamp);
       if (mappedFile != null) {
           long offset = 0;
           int low = minLogicOffset > mappedFile.getFileFromOffset() ? (int) (minLogicOffset - mappedFile.getFileFromOffset()) : 0;
           int high = 0;
           int midOffset = -1, targetOffset = -1, leftOffset = -1, rightOffset = -1;
           long leftIndexValue = -1L, rightIndexValue = -1L;
           long minPhysicOffset = this.defaultMessageStore.getMinPhyOffset();
           SelectMappedBufferResult sbr = mappedFile.selectMappedBuffer(0);
           if (null != sbr) {
               ByteBuffer byteBuffer = sbr.getByteBuffer();
               high = byteBuffer.limit() - CQ_STORE_UNIT_SIZE;
               try {
                   while (high >= low) {
                       midOffset = (low + high) / (2 * CQ_STORE_UNIT_SIZE) * CQ_STORE_UNIT_SIZE;
                       byteBuffer.position(midOffset);
                       long phyOffset = byteBuffer.getLong();
                       int size = byteBuffer.getInt();
                       if (phyOffset < minPhysicOffset) {
                           low = midOffset + CQ_STORE_UNIT_SIZE;
                           leftOffset = midOffset;
                           continue;
                       }

                       long storeTime =
                           this.defaultMessageStore.getCommitLog().pickupStoreTimestamp(phyOffset, size);
                       if (storeTime < 0) {
                           return 0;
                       } else if (storeTime == timestamp) {
                           targetOffset = midOffset;
                           break;
                       } else if (storeTime > timestamp) {
                           high = midOffset - CQ_STORE_UNIT_SIZE;
                           rightOffset = midOffset;
                           rightIndexValue = storeTime;
                       } else {
                           low = midOffset + CQ_STORE_UNIT_SIZE;
                           leftOffset = midOffset;
                           leftIndexValue = storeTime;
                       }
                   }

                   if (targetOffset != -1) {

                       offset = targetOffset;
                   } else {
                       if (leftIndexValue == -1) {

                           offset = rightOffset;
                       } else if (rightIndexValue == -1) {

                           offset = leftOffset;
                       } else {
                           offset =
                               Math.abs(timestamp - leftIndexValue) > Math.abs(timestamp
                                   - rightIndexValue) ? rightOffset : leftOffset;
                       }
                   }

                   return (mappedFile.getFileFromOffset() + offset) / CQ_STORE_UNIT_SIZE;
               } finally {
                   sbr.release();
               }
           }
       }
       return 0;
   }