浅析RocketMQ-CommitLog异步刷盘

CommitLog 分为同步刷盘和异步刷盘刷盘，默认情况都是异步刷盘。本篇也是以异步刷盘进行介绍。

前文介绍过，消息通过SendMessageProcessor进行处理，会调用submitFlushRequest唤醒对应刷盘服务

  public CompletableFuture<PutMessageStatus> submitFlushRequest(AppendMessageResult result, MessageExt messageExt) {
        // 同步刷盘
        if (FlushDiskType.SYNC_FLUSH == this.defaultMessageStore.getMessageStoreConfig().getFlushDiskType()) {
            final GroupCommitService service = (GroupCommitService) this.flushCommitLogService;
            if (messageExt.isWaitStoreMsgOK()) {
                GroupCommitRequest request = new GroupCommitRequest(result.getWroteOffset() + result.getWroteBytes(),
                        this.defaultMessageStore.getMessageStoreConfig().getSyncFlushTimeout());
                flushDiskWatcher.add(request);
                service.putRequest(request);
                return request.future();
            } else {
                service.wakeup();
                return CompletableFuture.completedFuture(PutMessageStatus.PUT_OK);
            }
        }
        // 异步刷盘
        else {
        	// 未开启堆外内存，直接执行flush操作
            if (!this.defaultMessageStore.getMessageStoreConfig().isTransientStorePoolEnable()) {
                flushCommitLogService.wakeup();
            } else  {
            	// 开启堆外内存配置，先将数据存到堆外内存中，执行commit操作
                commitLogService.wakeup();
            }
            return CompletableFuture.completedFuture(PutMessageStatus.PUT_OK);
        }
    }

异步刷盘分支区分在是否先将数据存入缓存区，放入缓存区是为了减少IO提升效率

一. Commit

commit操作是通过CommitRealTimeService实现的，在构造CommitLog对象时进行了初始化该值。FlushCommitLogService 实现了Runnable接口，在broker启动时调用。

 class CommitRealTimeService extends FlushCommitLogService {
		// 上次commit时间
        private long lastCommitTimestamp = 0;
        
        public void run() {
            while (!this.isStopped()) {
            	// 堵塞等待时间,默认200ms
                int interval = CommitLog.this.defaultMessageStore.getMessageStoreConfig().getCommitIntervalCommitLog();
				// 最少commit页数,默认 4页
                int commitDataLeastPages = CommitLog.this.defaultMessageStore.getMessageStoreConfig().getCommitCommitLogLeastPages();
				// 强制commit间隔,默认200ms
                int commitDataThoroughInterval =
                    CommitLog.this.defaultMessageStore.getMessageStoreConfig().getCommitCommitLogThoroughInterval();

                long begin = System.currentTimeMillis();
                // 上次commit时间+ commit间隔过长,则将未commmit数据都commit掉
                if (begin >= (this.lastCommitTimestamp + commitDataThoroughInterval)) {
                    this.lastCommitTimestamp = begin;
                    commitDataLeastPages = 0;
                }

                try {
                	// commit数据
                    boolean result = CommitLog.this.mappedFileQueue.commit(commitDataLeastPages);
                    if (!result) {
                        this.lastCommitTimestamp = end; 
                        // 唤醒flush服务
                        flushCommitLogService.wakeup();
                    }
                    // 继续堵塞等待
                    this.waitForRunning(interval);
                } catch (Throwable e) {}
            }
			
            boolean result = false;
            // 退出服务将，剩余的数据都commit掉
            for (int i = 0; i < RETRY_TIMES_OVER && !result; i++) {
                result = CommitLog.this.mappedFileQueue.commit(0);
            }
        }
    }

commit操作首先是查找上次commit指针的文件，然后执行commit操作，最后更新commit指针

    public boolean commit(final int commitLeastPages) {
        boolean result = true;
        // 找到上次commit位置的文件
        MappedFile mappedFile = this.findMappedFileByOffset(this.committedWhere, this.committedWhere == 0);
        if (mappedFile != null) {
        	// commit数据
            int offset = mappedFile.commit(commitLeastPages);
            // 更新commit的位置，offset返回的是在mappedFile中的偏移，committedWhere要记录总的偏移
            long where = mappedFile.getFileFromOffset() + offset;
            result = where == this.committedWhere;
            this.committedWhere = where;
        }

        return result;
    }

    public int commit(final int commitLeastPages) {
    	// writeBuffer为null说明没有开启堆外内存，故没有commit操作，直接返回
        if (writeBuffer == null) {
            return this.wrotePosition.get();
        }
        // isAbleToCommit满足以下任意条件即返回true
        // 1.文件写满了
        // 2.commitLeastPages>0,待commit数据是否达到最少页数
        // 3.commitLeastPages=0时，存在待commit数据
        if (this.isAbleToCommit(commitLeastPages)) {
            if (this.hold()) {
            	// 将数据写入fileChannel中，并更新commit指针
                commit0();
                this.release();
            } else {}
        }

        // 数据commit指针到文件最大值，则将mappedfile绑定的堆外内存归还
        if (writeBuffer != null && this.transientStorePool != null && this.fileSize == this.committedPosition.get()) {
            this.transientStorePool.returnBuffer(writeBuffer);
            this.writeBuffer = null;
        }

        return this.committedPosition.get();
    }

归纳整个流程，就是commit服务每隔200ms，查找可commit的数据.如果存在数据则将数据commit到filechannel,并通知flush服务执行。

二. Flush

Flush操作在异步刷盘情况下是通过FlushRealTimeService实现的，也是在构造CommitLog对象时进行了初始化该值。

class FlushRealTimeService extends FlushCommitLogService {
        private long lastFlushTimestamp = 0;
        private long printTimes = 0;

        public void run() {
            while (!this.isStopped()) {
            	// 是否定时刷盘，默认true
                boolean flushCommitLogTimed = CommitLog.this.defaultMessageStore.getMessageStoreConfig().isFlushCommitLogTimed();
				// 堵塞间隔500ms
                int interval = CommitLog.this.defaultMessageStore.getMessageStoreConfig().getFlushIntervalCommitLog();
                // 刷盘最少页数，默认4页
                int flushPhysicQueueLeastPages = CommitLog.this.defaultMessageStore.getMessageStoreConfig().getFlushCommitLogLeastPages();
				// 强制flush间隔，默认10s
                int flushPhysicQueueThoroughInterval =
                    CommitLog.this.defaultMessageStore.getMessageStoreConfig().getFlushCommitLogThoroughInterval();

                boolean printFlushProgress = false;

                // 距离上次flush过久，强制flush一次
                long currentTimeMillis = System.currentTimeMillis();
                if (currentTimeMillis >= (this.lastFlushTimestamp + flushPhysicQueueThoroughInterval)) {
                    this.lastFlushTimestamp = currentTimeMillis;
                    flushPhysicQueueLeastPages = 0;
                    printFlushProgress = (printTimes++ % 10) == 0;
                }

                try {
                	// 不同的休眠方式
                    if (flushCommitLogTimed) {
                        Thread.sleep(interval);
                    } else {
                        this.waitForRunning(interval);
                    }
                    // flush数据
                    CommitLog.this.mappedFileQueue.flush(flushPhysicQueueLeastPages);
                    long storeTimestamp = CommitLog.this.mappedFileQueue.getStoreTimestamp();
                    // 更新安全检查点的刷盘时间,用于重新启动broker时恢复数据
                    if (storeTimestamp > 0) {
                        CommitLog.this.defaultMessageStore.getStoreCheckpoint().setPhysicMsgTimestamp(storeTimestamp);
                    }
                } catch (Throwable e) {}
            }

            // 将未flush数据，全部flush到硬盘
            boolean result = false;
            for (int i = 0; i < RETRY_TIMES_OVER && !result; i++) {
                result = CommitLog.this.mappedFileQueue.flush(0);
            }
        }

    }

flush方法和commit类似，都是先找文件，然后flush数据，最后更新对应指针

    public boolean flush(final int flushLeastPages) {
        boolean result = true;
        // 查找上次flush的文件
        MappedFile mappedFile = this.findMappedFileByOffset(this.flushedWhere, this.flushedWhere == 0);
        if (mappedFile != null) {
            long tmpTimeStamp = mappedFile.getStoreTimestamp();
            // 刷盘
            int offset = mappedFile.flush(flushLeastPages);
            // 更新刷盘指针
            long where = mappedFile.getFileFromOffset() + offset;
            result = where == this.flushedWhere;
            this.flushedWhere = where;
            if (0 == flushLeastPages) {
                this.storeTimestamp = tmpTimeStamp;
            }
        }

        return result;
    }

    public int flush(final int flushLeastPages) {
    	// isAbleToFlush和isAbleToCommit操作类似，校验是否可以flush
        if (this.isAbleToFlush(flushLeastPages)) {
            if (this.hold()) {
            	// 这里开启堆外内存，则获取到commit指针，否则获取文件的写入位置
                int value = getReadPosition();

                try {
                    // 刷盘操作
                    if (writeBuffer != null || this.fileChannel.position() != 0) {
                        this.fileChannel.force(false);
                    } else {
                        this.mappedByteBuffer.force();
                    }
                } catch (Throwable e) { }
				// 更新flush指针
                this.flushedPosition.set(value);
                this.release();
            } else {}
        }
        return this.getFlushedPosition();
    }

归纳整个流程，就是flush服务每隔500ms，查找可flush的数据.如果存在数据则将刷盘数据，更新刷盘指针。如果距离上次刷盘超过10s则强制刷盘一次。