rocketmq writeQueueNum和readQueueNum理解-part2

rocketmq：4.3.2

测试环境一个小集群：2m, 2s

1.消息消费的入口，简单示例代码如下。

DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("group_name_4"); //step1
consumer.setNamesrvAddr(nameserver);
consumer.setConsumeFromWhere(ConsumeFromWhere.CONSUME_FROM_FIRST_OFFSET);
consumer.subscribe("topic_store_test4", "*"); //step2
consumer.registerMessageListener(new MessageListenerConcurrently() {
    @Override
    public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs,
                                                            ConsumeConcurrentlyContext context) {
        System.out.printf("%s Receive New Messages: %s %n", Thread.currentThread().getName(), msgs);
        return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
    }
});
consumer.start(); //step3
System.out.printf("Consumer Started.%n");
TimeUnit.MINUTES.sleep(10L);

具体流程说明：

1.1 在step1处，实际new的对象是DefaultMQPushConsumerImpl，它里面有一个变量，

private final RebalanceImpl rebalanceImpl = new RebalancePushImpl(this);

1.2 在这个类的父类里面有一个成员变量，

protected final ConcurrentMap<String/* topic */, Set<MessageQueue>> topicSubscribeInfoTable =
        new ConcurrentHashMap<String, Set<MessageQueue>>();

1.3 继续往下到step3，在start方法的内部，倒数第4个方法, 概要流程如下,关注step4

start() {
// do xxx
mQClientFactory.start(); //step3.5
this.updateTopicSubscribeInfoWhenSubscriptionChanged(); //step4
this.mQClientFactory.checkClientInBroker();
this.mQClientFactory.sendHeartbeatToAllBrokerWithLock();
this.mQClientFactory.rebalanceImmediately();
}

1.4.1 在step3.5处，实际调用了MQClientInstance.start(), 同时有一个任务线程RebalanceService也被启动了（这个是干队列负载均衡的）, 而最终进行负载均衡操作的地方又被委托到了 RebalanceImpl这个类里面, 即上面1.1 提到的。然后最后一顿操作后，会创建PullRequest的一个list，这些对象被添加进了PullMessageService对象的pullRequestQueue（一个阻塞队列）.

1.4.2 PullMessageService这个也是一个线程任务，在run方法里面，只有能够从pullRequestQueue拿到任务时，才会去broker实际的拉消息.

1.4.3 在step4的方法里一直向下debug，最终进入updateTopicRouteInfoFromNameServer方法，贴下代码(有点长)

public boolean updateTopicRouteInfoFromNameServer(final String topic, boolean isDefault,
        DefaultMQProducer defaultMQProducer) {
        try {
            if (this.lockNamesrv.tryLock(LOCK_TIMEOUT_MILLIS, TimeUnit.MILLISECONDS)) {
                try {
                    TopicRouteData topicRouteData;
                    if (isDefault && defaultMQProducer != null) {
                        topicRouteData = this.mQClientAPIImpl.getDefaultTopicRouteInfoFromNameServer(defaultMQProducer.getCreateTopicKey(),
                            1000 * 3);
                        if (topicRouteData != null) {
                            for (QueueData data : topicRouteData.getQueueDatas()) {
                                int queueNums = Math.min(defaultMQProducer.getDefaultTopicQueueNums(), data.getReadQueueNums());
                                data.setReadQueueNums(queueNums);
                                data.setWriteQueueNums(queueNums);
                            }
                        }
                    } else {
                        topicRouteData = this.mQClientAPIImpl.getTopicRouteInfoFromNameServer(topic, 1000 * 3);
                    }
                    if (topicRouteData != null) {
                        TopicRouteData old = this.topicRouteTable.get(topic);
                        boolean changed = topicRouteDataIsChange(old, topicRouteData);
                        if (!changed) {
                            changed = this.isNeedUpdateTopicRouteInfo(topic);
                        } else {
                            log.info("the topic[{}] route info changed, old[{}] ,new[{}]", topic, old, topicRouteData);
                        }

                        if (changed) {
                            TopicRouteData cloneTopicRouteData = topicRouteData.cloneTopicRouteData();

                            for (BrokerData bd : topicRouteData.getBrokerDatas()) {
                                this.brokerAddrTable.put(bd.getBrokerName(), bd.getBrokerAddrs());
                            }

                            // Update Pub info
                            {
                                TopicPublishInfo publishInfo = topicRouteData2TopicPublishInfo(topic, topicRouteData);
                                publishInfo.setHaveTopicRouterInfo(true);
                                Iterator<Entry<String, MQProducerInner>> it = this.producerTable.entrySet().iterator();
                                while (it.hasNext()) {
                                    Entry<String, MQProducerInner> entry = it.next();
                                    MQProducerInner impl = entry.getValue();
                                    if (impl != null) {
                                        impl.updateTopicPublishInfo(topic, publishInfo); 
                                    }
                                }
                            }

                            // Update sub info
                            {
                                Set<MessageQueue> subscribeInfo = topicRouteData2TopicSubscribeInfo(topic, topicRouteData);
                                Iterator<Entry<String, MQConsumerInner>> it = this.consumerTable.entrySet().iterator();
                                while (it.hasNext()) {
                                    Entry<String, MQConsumerInner> entry = it.next();
                                    MQConsumerInner impl = entry.getValue();
                                    if (impl != null) {
                                        impl.updateTopicSubscribeInfo(topic, subscribeInfo); // step5
                                    }
                                }
                            }
                            log.info("topicRouteTable.put. Topic = {}, TopicRouteData[{}]", topic, cloneTopicRouteData);
                            this.topicRouteTable.put(topic, cloneTopicRouteData);
                            return true;
                        }
                    } else {
                        log.warn("updateTopicRouteInfoFromNameServer, getTopicRouteInfoFromNameServer return null, Topic: {}", topic);
                    }
                } catch (Exception e) {
                    if (!topic.startsWith(MixAll.RETRY_GROUP_TOPIC_PREFIX) && !topic.equals(MixAll.AUTO_CREATE_TOPIC_KEY_TOPIC)) {
                        log.warn("updateTopicRouteInfoFromNameServer Exception", e);
                    }
                } finally {
                    this.lockNamesrv.unlock();
                }
            } else {
                log.warn("updateTopicRouteInfoFromNameServer tryLock timeout {}ms", LOCK_TIMEOUT_MILLIS);
            }
        } catch (InterruptedException e) {
            log.warn("updateTopicRouteInfoFromNameServer Exception", e);
        }

        return false;
    }

在注释 // Update sub info  处，会根据readQueueNum的数量 * broker的数量，形成一个MessageQueue的集合。

然后在step5处，会把在1.2 处提到的变量(topicSubscribeInfoTable)赋值. 这样在1.4.2 处，才会实际去拉消息

 

2.服务端处理客户端拉取消息的基本操作

在broker模块, 接收到client端(最终进行发送请求的地方：NettyRemotingAbstract.invokeSyncImpl方法)发送的pull消息的请求的处理类是：PullMessageProcessor.processRequest。

根据上篇(https://blog.csdn.net/shuipinglp/article/details/101058449)的验证case-2. 

当writeQueueNum=2， readQueueNum=4时，写入消息生成了2个consumeQueue文件，

在往下就是读取消息的代码位置了。具体看一下就好，不复杂。

final GetMessageResult getMessageResult =
            this.brokerController.getMessageStore().getMessage(requestHeader.getConsumerGroup(), requestHeader.getTopic(),
                requestHeader.getQueueId(), requestHeader.getQueueOffset(), requestHeader.getMaxMsgNums(), messageFilter);

此时getMessageResult的数据类似如下：

GetMessageResult [status=NO_MESSAGE_IN_QUEUE, nextBeginOffset=0, minOffset=0, maxOffset=0, bufferTotalSize=0, suggestPullingFromSlave=false]。

具体的调用流程如下

PullRequestHoldService(任务线程).run方法 --> 然后扔一个异步任务

this.brokerController.getPullMessageProcessor().executeRequestWhenWakeup(request.getClientChannel(),
                                request.getRequestCommand());

流程在回到PullMessageProcessor.processRequest，进入到代码下面的一段switch语句

switch (getMessageResult.getStatus()) {
                case FOUND:
                    response.setCode(ResponseCode.SUCCESS);
                    break;
                case MESSAGE_WAS_REMOVING:
                    response.setCode(ResponseCode.PULL_RETRY_IMMEDIATELY);
                    break;
                case NO_MATCHED_LOGIC_QUEUE:
                case NO_MESSAGE_IN_QUEUE: 
                    if (0 != requestHeader.getQueueOffset()) {
                        response.setCode(ResponseCode.PULL_OFFSET_MOVED);

                        // XXX: warn and notify me
                        log.info("the broker store no queue data, fix the request offset {} to {}, Topic: {} QueueId: {} Consumer Group: {}",
                            requestHeader.getQueueOffset(),
                            getMessageResult.getNextBeginOffset(),
                            requestHeader.getTopic(),
                            requestHeader.getQueueId(),
                            requestHeader.getConsumerGroup()
                        );
                    } else {
                        response.setCode(ResponseCode.PULL_NOT_FOUND);
                    }
                    break;
                case NO_MATCHED_MESSAGE:
//省略

在case NO_MESSAGE_IN_QUQUE处，response code被设置成了PULL_NOT_FOUND. 

接下来看客户端拉取消息处的处理逻辑：

调用发起的地方：

DefaultMQPushConsumerImpl.pullMessage 调用

PullAPIWrapper.pullKernelImpl, （同时传递了一个PullCallback）继续 调用

MQClientAPIImpl.pullMessage, 然后方法processPullResponse，会处理返回的结果。部分代码

switch (response.getCode()) {
            case ResponseCode.SUCCESS:
                pullStatus = PullStatus.FOUND;
                break;
            case ResponseCode.PULL_NOT_FOUND:
                pullStatus = PullStatus.NO_NEW_MSG;
                break;
            case ResponseCode.PULL_RETRY_IMMEDIATELY:
                pullStatus = PullStatus.NO_MATCHED_MSG;
                break;
            case ResponseCode.PULL_OFFSET_MOVED:
                pullStatus = PullStatus.OFFSET_ILLEGAL;
                break;

            default:
                throw new MQBrokerException(response.getCode(), response.getRemark());
        }

最终把pullStatus设置成了NO_NEW_MSG.

消息获取之后，触发上面提到的PullCalllback，部分代码如下

case NO_NEW_MSG:
                            pullRequest.setNextOffset(pullResult.getNextBeginOffset());

                            DefaultMQPushConsumerImpl.this.correctTagsOffset(pullRequest);

                            DefaultMQPushConsumerImpl.this.executePullRequestImmediately(pullRequest);
                            break;

重置了一下相关的offset，然后PullRequest继续入队列，继续从broker拉消息（其实啥也拉不到）。

 

3.当writeQueueNum=2， readQueueNum=4，消息是可以全部获取的， 但是在rebalance的时候会分配到queueId=2和3的情况，客端户是没有报错发生。

备注：可以通过RequestCode里面的code码进行查找客户端和服务器的处理对应关系，里面的码值在加上注释就更好了。