RocketMQ消息存储二：数据文件存组件储层

数据文件存储组件层只有一个类DefaultMessageStore，它负责所有存储相关的工作，上层业务中的消息发送处理、查询处理、拉取处理，最终都委托给DefaultMessageStore操作。

核心属性

    // 存储配置
    private final MessageStoreConfig messageStoreConfig;
    // CommitLog原始消息存储
    private final CommitLog commitLog;
    // 消费队列信息
    private final ConcurrentMap<String/* topic */, ConcurrentMap<Integer/* queueId */, ConsumeQueue>> consumeQueueTable;
    // ConsumeQueue刷盘
    private final FlushConsumeQueueService flushConsumeQueueService;
    // 清理commitLog
    private final CleanCommitLogService cleanCommitLogService;
    // 清理ConsumeQueue刷
    private final CleanConsumeQueueService cleanConsumeQueueService;
    // 消息索引服务
    private final IndexService indexService;
    // 新建 MappedFile 服务
    private final AllocateMappedFileService allocateMappedFileService;
    // 消息分发
    private final ReputMessageService reputMessageService;
    // 主从同步高可用
    private final HAService haService;
    // 定时消息
    private final ScheduleMessageService scheduleMessageService;
    // 统计服务
    private final StoreStatsService storeStatsService;
    // 短暂存储池
    private final TransientStorePool transientStorePool;

    private final RunningFlags runningFlags = new RunningFlags();
    private final SystemClock systemClock = new SystemClock();
    // 定时任务
    private final ScheduledExecutorService scheduledExecutorService =
        Executors.newSingleThreadScheduledExecutor(new ThreadFactoryImpl("StoreScheduledThread"));
    private final BrokerStatsManager brokerStatsManager;
    private final MessageArrivingListener messageArrivingListener;
    private final BrokerConfig brokerConfig;

    private volatile boolean shutdown = true;

    private StoreCheckpoint storeCheckpoint;

    private AtomicLong printTimes = new AtomicLong(0);

    private final LinkedList<CommitLogDispatcher> dispatcherList;
    // 文件 
    private RandomAccessFile lockFile;

内部方法

start()方法

start方法是启动入口，主要逻辑为：（1）加锁，一个broker只能启动一个DefaultMessageStore；（2）CommitLog、ConsumeQueue检查与恢复；（3）启动DefaultMessageStore内的各种服务。

putMessage()、putMessages()方法

存储消息的方法，逻辑比较简单，首先检查是否可写，接着将发送消息委托给CommitLog处理。最后统计耗时情况和失败情况。

isOSPageCacheBusy()判断系统是否繁忙

    public boolean isOSPageCacheBusy() {
   
        // putMessage 加锁的时间，如果没有正在写入，则值为0
        long begin = this.getCommitLog().getBeginTimeInLock();
        long diff = this.systemClock.now() - begin;
        // 超过配置时间，则为系统繁忙
        return diff < 10000000
                && diff > this.messageStoreConfig.getOsPageCacheBusyTimeOutMills();
    }

getMessage() 获取消息

逻辑见注释

    public GetMessageResult getMessage(final String group, final String topic, final int queueId, final long offset,
        final int maxMsgNums,
        final MessageFilter messageFilter) {
   

        // 前置校验，省略

        // 获取ConsumeQueue
        ConsumeQueue consumeQueue = findConsumeQueue(topic, queueId);
        if (consumeQueue != null) {
   
            // 检查offset合法性
            minOffset = consumeQueue.getMinOffsetInQueue();
            maxOffset = consumeQueue.getMaxOffsetInQueue();

            if (maxOffset == 0) {
   
                status = GetMessageStatus.NO_MESSAGE_IN_QUEUE;
                nextBeginOffset = nextOffsetCorrection(offset, 0);
            } else if (offset < minOffset) {
   
                status = GetMessageStatus.OFFSET_TOO_SMALL;
                nextBeginOffset = nextOffsetCorrection(offset, minOffset);
            } else if (offset == maxOffset) {
   
                status = GetMessageStatus.OFFSET_OVERFLOW_ONE;
                nextBeginOffset = nextOffsetCorrection(offset, offset);
            } else if (offset > maxOffset) {
   
                status = GetMessageStatus.OFFSET_OVERFLOW_BADLY;
                if (0 == minOffset) {
   
                    nextBeginOffset = nextOffsetCorrection(offset, minOffset);
                } else {
   
                    nextBeginOffset = nextOffsetCorrection(offset, maxOffset);
                }
            } else {
   
                // offset合法，从ConsumeQueue中获取可读的bytebuffer
                SelectMappedBufferResult bufferConsumeQueue = consumeQueue.getIndexBuffer(offset);
                if (bufferConsumeQueue != null) {
   
                    try {
   
                        status = GetMessageStatus.NO_MATCHED_MESSAGE;
                        long nextPhyFileStartOffset = Long.MIN_VALUE;
                        long maxPhyOffsetPulling = 0;

                        int i= 0;
                        final int maxFilterMessageCount = Math.max(16000, maxMsgNums * ConsumeQueue.CQ_STORE_UNIT_SIZE);
                        final boolean diskFallRecorded = this.messageStoreConfig.isDiskFallRecorded();
                        ConsumeQueueExt.CqExtUnit cqExtUnit = new ConsumeQueueExt.CqExtUnit();
                        // 遍历bytebuffer，对每条消息进程处理。ConsumeQueue消息定长，每个20字节
                        for (; i < bufferConsumeQueue.getSize() && i < maxFilterMessageCount; i += ConsumeQueue.CQ_STORE_UNIT_SIZE) {
   
                            long offsetPy = bufferConsumeQueue.getByteBuffer().getLong();
                            int sizePy = bufferConsumeQueue.getByteBuffer().getInt();
                            long tagsCode = bufferConsumeQueue.getByteBuffer().getLong();

                            maxPhyOffsetPulling = offsetPy;

                            if (nextPhyFileStartOffset != Long.MIN_VALUE) {
   
                                if (offsetPy < nextPhyFileStartOffset)
                                    continue;
                            }
                            // 消息在磁盘
                            boolean isInDisk = checkInDiskByCommitOffset(offsetPy, maxOffsetPy);
                            // 此批消息拉取到上限，结束
                            if (