rocketmq核心源码分析第九篇一消息处理第三部分一broker通信层接收消息

消息处理模型

启动阶段: 消息处理类

• 发送消息 和发送批量消息都交由SendMessageProcessor处理
  

运行阶段: 请求响应映射

• 同一个通道可能会发送多个消息,请求与响应一一对应
• 通过 opaque完成req-rep映射模型

业务层发送消息
客户端通信层构建req-rep唯一id opaque,以及future等待响应并发送消息
服务端通信层设置rep唯一id为req唯一id [opaque],并响应
客户端通信层根据rep 唯一id查找future池中的相应future对象
取消future阻塞,唤醒业务层线程

源码分析一SendMessageProcessor.asyncProcessRequest

• 消息消费失败回发给broker的消息处理
• 消息轨迹处理
• 批量消息处理
• 消息处理

public CompletableFuture<RemotingCommand> asyncProcessRequest(ChannelHandlerContext ctx,
                                                                  RemotingCommand request) throws RemotingCommandException {
    final SendMessageContext mqtraceContext;
    switch (request.getCode()) {
        消息消费失败回发给broker的消息
        case RequestCode.CONSUMER_SEND_MSG_BACK:
            return this.asyncConsumerSendMsgBack(ctx, request);
        default:
            SendMessageRequestHeader requestHeader = parseRequestHeader(request);
            if (requestHeader == null) {
                return CompletableFuture.completedFuture(null);
            }
            消息轨迹
            mqtraceContext = buildMsgContext(ctx, requestHeader);
            this.executeSendMessageHookBefore(ctx, request, mqtraceContext);
            处理批量消息
            if (requestHeader.isBatch()) {
                return this.asyncSendBatchMessage(ctx, request, mqtraceContext, requestHeader);
            } else {
                异步处理消息
                return this.asyncSendMessage(ctx, request, mqtraceContext, requestHeader);
            }
        }
    }    
}   

源码分析一SendMessageProcessor.asyncSendMessage

• 重试达上限构建死信队列 根据消息头获取重试的次数
• 判断是否允许事务消息存储【参见后续事务消息章节】
• 调用TransactionalMessageService完成事务消息转换存储
• 普通消息存储

private CompletableFuture<RemotingCommand> asyncSendMessage(ChannelHandlerContext ctx, RemotingCommand request,
                                                                SendMessageContext mqtraceContext,
                                                                SendMessageRequestHeader requestHeader) {
            
        ...... 删除其他代码
        MessageExtBrokerInner msgInner = new MessageExtBrokerInner();
        msgInner.setTopic(requestHeader.getTopic());
        msgInner.setQueueId(queueIdInt);
        处理重试或者死信队列 根据消息头获取重试的次数
        if (!handleRetryAndDLQ(requestHeader, response, request, msgInner, topicConfig)) {
            return CompletableFuture.completedFuture(response);
        }

       
        ...... 删除其他代码
        CompletableFuture<PutMessageResult> putMessageResult = null;
        Map<String, String> origProps = MessageDecoder.string2messageProperties(requestHeader.getProperties());

        获取producer发送的时候设置的事务消息属性[prepare消息  commit消息应该算是普通消息]
        String transFlag = origProps.get(MessageConst.PROPERTY_TRANSACTION_PREPARED);
        判断是否允许事务消息存储
        if (transFlag != null && Boolean.parseBoolean(transFlag)) {
            if (this.brokerController.getBrokerConfig().isRejectTransactionMessage()) {
                response.setCode(ResponseCode.NO_PERMISSION);
                response.setRemark(
                        "the broker[" + this.brokerController.getBrokerConfig().getBrokerIP1()
                                + "] sending transaction message is forbidden");
                return CompletableFuture.completedFuture(response);
            }
            调用TransactionalMessageService完成事务消息转换存储
            putMessageResult = this.brokerController.getTransactionalMessageService().asyncPrepareMessage(msgInner);
        } else {
            存储消息
            putMessageResult = this.brokerController.getMessageStore().asyncPutMessage(msgInner);
        }
        返回
        return handlePutMessageResultFuture(putMessageResult, response, request, msgInner, responseHeader, mqtraceContext, ctx, queueIdInt);
    }

源码分析一handleRetryAndDLQ

• 重试转死信时确保死信队列存在
• 判断是否消费者消费失败发回的重试消息
• 判断重试消息是否达最大消费次数
• 创建或获取的topic信息,确保死信topic存在

	private boolean handleRetryAndDLQ(SendMessageRequestHeader requestHeader, RemotingCommand response,
                                  RemotingCommand request,
                                  MessageExt msg, TopicConfig topicConfig) {                       
        String newTopic = requestHeader.getTopic();
        判断是否消费者消费失败发回的重试消息
        if (null != newTopic && newTopic.startsWith(MixAll.RETRY_GROUP_TOPIC_PREFIX)) {
            String groupName = newTopic.substring(MixAll.RETRY_GROUP_TOPIC_PREFIX.length());
            SubscriptionGroupConfig subscriptionGroupConfig =
                this.brokerController.getSubscriptionGroupManager().findSubscriptionGroupConfig(groupName);
            if (null == subscriptionGroupConfig) {
                response.setCode(ResponseCode.SUBSCRIPTION_GROUP_NOT_EXIST);
                response.setRemark(
                    "subscription group not exist, " + groupName + " " + FAQUrl.suggestTodo(FAQUrl.SUBSCRIPTION_GROUP_NOT_EXIST));
                return false;
            }
            区分下版本号
            int maxReconsumeTimes = subscriptionGroupConfig.getRetryMaxTimes();
            if (request.getVersion() >= MQVersion.Version.V3_4_9.ordinal()) {
                maxReconsumeTimes = requestHeader.getMaxReconsumeTimes();
            }
            判断重试消息是否达最大消费次数
            int reconsumeTimes = requestHeader.getReconsumeTimes() == null ? 0 : requestHeader.getReconsumeTimes();
            if (reconsumeTimes >= maxReconsumeTimes) {
                newTopic = MixAll.getDLQTopic(groupName);
                int queueIdInt = Math.abs(this.random.nextInt() % 99999999) % DLQ_NUMS_PER_GROUP;
                // 创建或获取存在的topic信息
                topicConfig = this.brokerController.getTopicConfigManager().createTopicInSendMessageBackMethod(newTopic,
                    DLQ_NUMS_PER_GROUP,
                    PermName.PERM_WRITE, 0
                );
                msg.setTopic(newTopic);
                msg.setQueueId(queueIdInt);
                if (null == topicConfig) {
                    response.setCode(ResponseCode.SYSTEM_ERROR);
                    response.setRemark("topic[" + newTopic + "] not exist");
                    return false;
                }
            }
        }
        int sysFlag = requestHeader.getSysFlag();
        if (TopicFilterType.MULTI_TAG == topicConfig.getTopicFilterType()) {
            sysFlag |= MessageSysFlag.MULTI_TAGS_FLAG;
        }
        msg.setSysFlag(sysFlag);
        return true;
    }

总结

• 从消息发送的底层通信到达messagestore
• 其中同时处理了重试,死信,批量,单个,消费者重发等场景
• 注意此时并没有落盘