RocketMQ Broker 瞬时存储池TransientStorePool的复用、源码分析（04）

风吹脚趾

于 2024-01-12 10:01:43 发布

阅读量506

点赞数 8

分类专栏： RocketMQ源码解析、实战分享文章标签： rocketmq

本文链接：https://blog.csdn.net/qq_37867087/article/details/135507396

版权

RocketMQ源码解析、实战分享专栏收录该内容

3 篇文章 0 订阅

订阅专栏

1 创建commitedLog文件，和瞬时内存池建立映射

发送消息的时候需要获取队列映射文件MappedFile(CommitLog#asyncPutMessage)
新创建的文件或者文件已满需要创建新的文件 (MappedFileQueue#getLastMappedFile)
构建申请文件的请求，将请求存放到requestQueue队列中，等待指定线程创建映射文件Mapped. ile (AllocateMappedFileService#putRequestAndReturnMappedFile)
等待分配线程创建MappedFile，创建之后唤醒获取映射文件的线程，返回映射文件 ,存放入队列 MappedFileQueue

存放申请文件请求的时候，会等待申请线程创建映射文件，当映射文件创建完成时才会返回对应的映射文件，并将创建的映射文件存入MappedFileQueue队列中

 public MappedFile putRequestAndReturnMappedFile(String nextFilePath, String nextNextFilePath, int fileSize) {
        int canSubmitRequests = 2; // TODO 换算需要分配文件的请求数量
        if (this.messageStore.isTransientStorePoolEnable()) {
            if (this.messageStore.getMessageStoreConfig().isFastFailIfNoBufferInStorePool()
                && BrokerRole.SLAVE != this.messageStore.getMessageStoreConfig().getBrokerRole()) { //if broker is slave, don't fast fail even no buffer in pool
                canSubmitRequests = this.messageStore.remainTransientStoreBufferNumbs() - this.requestQueue.size(); //TODO 瞬时缓存中的buffer数量 - 请求队列中的数量 = 需要提交的请求
            }   // TODO 可以提交的请求最多不能超过 瞬时缓存池中的遗留的buffer数量
        }
        // TODO 下个文件的请求的 创建
        AllocateRequest nextReq = new AllocateRequest(nextFilePath, fileSize);
        boolean nextPutOK = this.requestTable.putIfAbsent(nextFilePath, nextReq) == null;

        if (nextPutOK) {
            if (canSubmitRequests <= 0) {
                log.warn("[NOTIFYME]TransientStorePool is not enough, so create mapped file error, " +
                    "RequestQueueSize : {}, StorePoolSize: {}", this.requestQueue.size(), this.messageStore.remainTransientStoreBufferNumbs());
                this.requestTable.remove(nextFilePath);
                return null;
            }
            boolean offerOK = this.requestQueue.offer(nextReq);
            if (!offerOK) {
                log.warn("never expected here, add a request to preallocate queue failed");
            }
            canSubmitRequests--;
        }
        // TODO 下下个文件的请求的 创建
        AllocateRequest nextNextReq = new AllocateRequest(nextNextFilePath, fileSize);
        boolean nextNextPutOK = this.requestTable.putIfAbsent(nextNextFilePath, nextNextReq) == null;
        if (nextNextPutOK) {
            if (canSubmitRequests <= 0) {
                log.warn("[NOTIFYME]TransientStorePool is not enough, so skip preallocate mapped file, " +
                    "RequestQueueSize : {}, StorePoolSize: {}", this.requestQueue.size(), this.messageStore.remainTransientStoreBufferNumbs());
                this.requestTable.remove(nextNextFilePath);
            } else {
                boolean offerOK = this.requestQueue.offer(nextNextReq);
                if (!offerOK) {
                    log.warn("never expected here, add a request to preallocate queue failed");
                }
            }
        }

        if (hasException) {
            log.warn(this.getServiceName() + " service has exception. so return null");
            return null;
        }
        // TODO 先获取第一个分配的文件，需要确定 AllocateRequest中mappedFile的初始化在线程中循环进行初始化创建文件和创建文件映射
        AllocateRequest result = this.requestTable.get(nextFilePath);
        try {
            if (result != null) {
                messageStore.getPerfCounter().startTick("WAIT_MAPFILE_TIME_MS");
                boolean waitOK = result.getCountDownLatch().await(waitTimeOut, TimeUnit.MILLISECONDS);  // TODO 设置等待创建对应的MappedFile文件，等待线程处理填充，然后唤醒继续返回对应的MappedFile
                messageStore.getPerfCounter().endTick("WAIT_MAPFILE_TIME_MS");
                if (!waitOK) {
                    log.warn("create mmap timeout " + result.getFilePath() + " " + result.getFileSize());
                    return null;
                } else {
                    this.requestTable.remove(nextFilePath);
                    return result.getMappedFile();
                }
            } else {
                log.error("find preallocate mmap failed, this never happen");
            }
        } catch (InterruptedException e) {
            log.warn(this.getServiceName() + " service has exception. ", e);
        }

        return null;
    }

线程轮训取出创建文件的请求进行创建文件