ROCKETMQ记录源码细节及疑问

 

 

记录相关细节：

Client模块：

1.DefaultMQPushConsumer.subscribe方法，一经调用如果MQClientInstance不为空则发送心跳给Broker

2.MessageModel是BROADCASTING的则使用LocalFileOffsetStore

CLUSTERING则使用RemoteBrokerOffsetStore

3.默认5秒钟持久化一次offset MQClientInstance.startScheduledTask line 308

4.默认30秒发送一次心跳消息 MQClientInstance.startScheduledTask line 296

5.默认2分钟获取一次namesrv地址 MQClientInstance.startScheduledTask line 270

6.默认30秒更新一次路由信息 即 消费者、生产者对应的Topic信息(具体信息 之后再细看) MQClientInstance.startScheduledTask line 285

7.默认30秒清空对应消费者没有使用到的broker信息 MQClientInstance.startScheduledTask line 296

8.默认1分钟调整一次线程池情况 (实际代码注释掉了)  MQClientInstance.startScheduledTask line 320

9.消费者线程池默认初始化线程数是20个，最大值是64  ConsumeMessageConcurrentlyService line 72

10.客户端会记录group+topic的消费次数 和 消费时间 ConsumeMessageConcurrentlyService line 459  StatsItem

11.消费失败，默认重新消费16次之后才放弃DefaultMQPushConsumerImpl line 523

12.消费失败, 消费者内部会间隔5秒钟之后重试一次 DefaultMQPushConsumerImpl line 309

        switch (this.defaultMQPushConsumer.getMessageModel()) {
            case BROADCASTING:
                for (int i = ackIndex + 1; i < consumeRequest.getMsgs().size(); i++) {
                    MessageExt msg = consumeRequest.getMsgs().get(i);
                    log.warn("BROADCASTING, the message consume failed, drop it, {}", msg.toString());
                }
                break;
            case CLUSTERING:
                List<MessageExt> msgBackFailed = new ArrayList<MessageExt>(consumeRequest.getMsgs().size());
                for (int i = ackIndex + 1; i < consumeRequest.getMsgs().size(); i++) {
                    MessageExt msg = consumeRequest.getMsgs().get(i);
                    boolean result = this.sendMessageBack(msg, context);
                    if (!result) {
                        msg.setReconsumeTimes(msg.getReconsumeTimes() + 1);
                        msgBackFailed.add(msg);
                    }
                }

                if (!msgBackFailed.isEmpty()) {
                    consumeRequest.getMsgs().removeAll(msgBackFailed);

                    this.submitConsumeRequestLater(msgBackFailed, consumeRequest.getProcessQueue(), consumeRequest.getMessageQueue());
                }
                break;
            default:
                break;
        }

13.消费失败 Client会回发信息给Broker  CONSUMER_SEND_MSG_BACK  PULL_MESSAGE

ConsumerSendMsgBackRequestHeader

PullMessageRequestHeader

问题：

消费者消费还没结束，就会已经在重新拉新一批消息了，orderly消费模式是怎么处理的?

因为消费者消费是提交线程池去处理的，是异步的，待消费信息都提交到线程池之后 就会提交新的拉取请求给 拉取的线程池等待拉取

 

Namesrv模块

10.Namesrv模块采用CommandLine方式处理命令, 且Options的longArgName的值会反写到nameSrvConfig和nettyServerConfig

11.Namesrv模块使用的是DefaultRequestProcessor NameSrvController line 151

12.每10秒钟扫描是否有不活跃的Broker NamesrvController line 93

13.BrokerLinveInfo的最后更新时间距离当前时间超过2分钟则判定是失效的Broker，进行移除  RouteInfoManager line 418

14.TlsSystemConfig 维护了所有TLS的jvm key

15.1秒钟获取一次请求返回的结果集  NettyRemotingService line 223

16.注册Broker信息 RouteInfoManager line 102

17.MQClientInstance.clientId产生逻辑

    private String clientIP = RemotingUtil.getLocalAddress();
    private String instanceName = System.getProperty("rocketmq.client.name", "DEFAULT");

    public String buildMQClientId() {
        StringBuilder sb = new StringBuilder();
        sb.append(this.getClientIP());

        sb.append("@");
        sb.append(this.getInstanceName());
        if (!UtilAll.isBlank(this.unitName)) {
            sb.append("@");
            sb.append(this.unitName);
        }

        return sb.toString();
    }

 

1.MessageListener实现最终返回null的话，默认是RECONSUME_LATER

            if (null == status) {
                log.warn("consumeMessage return null, Group: {} Msgs: {} MQ: {}",
                    ConsumeMessageConcurrentlyService.this.consumerGroup,
                    msgs,
                    messageQueue);
                status = ConsumeConcurrentlyStatus.RECONSUME_LATER;
            }

2.消费端、生产端针对Broker发送心跳只针对MASTER节点发送

MQClientInstance line 513

    private void sendHeartbeatToAllBroker() {
        final HeartbeatData heartbeatData = this.prepareHeartbeatData();
        final boolean producerEmpty = heartbeatData.getProducerDataSet().isEmpty();
        final boolean consumerEmpty = heartbeatData.getConsumerDataSet().isEmpty();
        if (producerEmpty && consum