RocketMQ存储之MappedFile

一、概述

RocketMQ的存储依赖于三类文件：CommitLog、ConsumeQueue、IndexFile，每类文件的具体作用可以参考官方文档，这里我们只做源码分析。三类存储文件都依赖MappedFile 来实现文件底层的数据读写。从类名可以看出：MappedFile 是通过内存映射文件来实现文件操作的，本篇我们分析MappedFile 的实现细节(version 4.8.0)。

二、实现细节

变量

我们先来看一下MappedFile 的变量：

    /**
     * 操作系统内存页大小
     */
    public static final int OS_PAGE_SIZE = 1024 * 4;
    protected static final InternalLogger log = InternalLoggerFactory.getLogger(LoggerName.STORE_LOGGER_NAME);

    private static final AtomicLong TOTAL_MAPPED_VIRTUAL_MEMORY = new AtomicLong(0);

    private static final AtomicInteger TOTAL_MAPPED_FILES = new AtomicInteger(0);

    /**
     * 写位置，文件写到buffer的position
     */
    protected final AtomicInteger wrotePosition = new AtomicInteger(0);

    /**
     * 提交位置，文件提交到文件映射的内存的位置
     */
    protected final AtomicInteger committedPosition = new AtomicInteger(0);

    /**
     * flush位置，文件刷盘的位置
     */
    private final AtomicInteger flushedPosition = new AtomicInteger(0);
    protected int fileSize;
    protected FileChannel fileChannel;
    /**
     * Message will put to here first, and then reput to FileChannel if writeBuffer is not null.
     * 如果writeBuffer不是空的话，消息会先写到临时的writeBuffer中，随后才会被写到fileChannel中。
     */
    protected ByteBuffer writeBuffer = null;

    /**
     * 临时存储池，即内存池
     */
    protected TransientStorePool transientStorePool = null;
    private String fileName;
    private long fileFromOffset;
    private File file;
    /**
     * 文件内存映射的buffer
     */
    private MappedByteBuffer mappedByteBuffer;
    private volatile long storeTimestamp = 0;
    private boolean firstCreateInQueue = false;

初始化

MappedFile的构造方法：

    public MappedFile() {
   
    }

    public MappedFile(final String fileName, final int fileSize) throws IOException {
   
        init(fileName, fileSize);
    }

    public MappedFile(final String fileName, final int fileSize,
        final TransientStorePool transientStorePool) throws IOException {
   
        init(fileName, fileSize, transientStorePool);
    }

两个有参构造方法需要需要调用init的两个重载方法来完成初始化。
init方法实现逻辑：

    public void init(final String fileName, final int fileSize,
        final TransientStorePool transientStorePool) throws IOException {
   
        init(fileName, fileSize);
        // 只有方法传入了transientStorePool存储池，成员变量writeBuffer才会被赋值
        this.writeBuffer = transientStorePool.borrowBuffer();
        this.transientStorePool = transientStorePool;
    }

    private void init(final String fileName, final int fileSize) throws IOException {
   
        this.fileName = fileName;
        this.fileSize = fileSize;
        this.file = new File(fileName);
        // 映射文件的第一个字节在组文件中的相对位置
        this.fileFromOffset = Long.parseLong(this.file.getName());
        boolean ok = false;

        ensureDirOK(this.file.getParent());

        try {
   
        	// 获取文件的channel
            this.fileChannel = new RandomAccessFile(this.file, "rw").getChannel();
            // 映射文件
            this.mappedByteBuffer = this.fileChannel.map(MapMode.READ_WRITE, 0, fileSize);
            TOTAL_MAPPED_VIRTUAL_MEMORY.addAndGet(fileSize);
            TOTAL_MAPPED_FILES.incrementAndGet();
            ok = true;
        } catch (FileNotFoundException e) {
   
            log.error("Failed to create file " + this.fileName, e);
            throw e;
        } catch (IOException e) {
   
            log.error("Failed to map file " + this.fileName, e);
            throw e;
        } finally {
   
            if (!ok && this.fileChannel != null) {
   
                this.fileChannel.close();
            }