Producer要发送Message,肯定需要先知道Topic路由信息,这样才能找到提供Topic服务的Broker master结点并发送Message。之前的文章Topic是如何创建和保存的,我们知道了Topic在Broker保存了一份,然后Broker定期向NameSrv注册Broker信息。这样NameSrv就存有所有Topic路由信息。因此Producer要发送消息,肯定需要从NameSrv获取路由信息,事实上Producer会定期从NameSrv获取Topic路由信息并存到内存中。
Producer端处理
在Producer的MQClientInstance启动后,会启动一系列定时任务
this.startScheduledTask();
其中在这个定时任务里面就包括定期(默认间隔30秒)向NameSrv更新Topic路由信息:
this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
@Override
public void run() {
try {
MQClientInstance.this.updateTopicRouteInfoFromNameServer();
} catch (Exception e) {
log.error("ScheduledTask updateTopicRouteInfoFromNameServer exception", e);
}
}
}, 10, this.clientConfig.getPollNameServerInterval(), TimeUnit.MILLISECONDS);
Producer获取Topic路由信息,是获取NameSrv上面所有的Topic路由信息吗?显然不合理,如果NameSrv上面有上万个Topic,那传输量就会很大,而且也没有那个必要。实际上,Producer只关心发送消息所需要的Topic的路由信息。
public void updateTopicRouteInfoFromNameServer() {
Set<String> topicList = new HashSet<String>();
// 省略了consumer的代码
// Producer 【1】
{
Iterator<Entry<String, MQProducerInner>> it = this.producerTable.entrySet().iterator();
while (it.hasNext()) {
Entry<String, MQProducerInner> entry = it.next();
MQProducerInner impl = entry.getValue();
if (impl != null) {
Set<String> lst = impl.getPublishTopicList();
topicList.addAll(lst);
}
}
}
for (String topic : topicList) { // 【2】
this.updateTopicRouteInfoFromNameServer(topic);
}
}
【1】中首先遍历producer,拿到所有的topic。然后【2】中拉取每个topic的路由信息并更新。
上面的代码看似很简单,但有个疑问:producer发送消息的时候才会指定topic,创建的时候并没有指定topic,那么更新时需要的topic列表是哪里来的?
首先先明确一点,Topic路由信息在producer端是存放在topicPublishInfoTable中的:
private final ConcurrentMap<String/* topic */, TopicPublishInfo> topicPublishInfoTable =
new ConcurrentHashMap<String, TopicPublishInfo>();
实际上,在发送消息的时候,会试图从producer的TopicPublishInfoTable中获取路由信息,如果没有,就会新增一个并立即向NameSrv发起更新Topic路由信息请求:
private TopicPublishInfo tryToFindTopicPublishInfo(final String topic) {
TopicPublishInfo topicPublishInfo = this.topicPublishInfoTable.get(topic);
if (null == topicPublishInfo || !topicPublishInfo.ok()) {
this.topicPublishInfoTable.putIfAbsent(topic, new TopicPublishInfo());
this.mQClientFactory.updateTopicRouteInfoFromNameServer(topic);
topicPublishInfo = this.topicPublishInfoTable.get(topic);
}
if (topicPublishInfo.isHaveTopicRouterInfo() || topicPublishInfo.ok()) {
return topicPublishInfo;
} else {
this.mQClientFactory.updateTopicRouteInfoFromNameServer(topic, true, this.defaultMQProducer);
topicPublishInfo = this.topicPublishInfoTable.get(topic);
return topicPublishInfo;
}
}
因此,定时任务定期从NameSrv拉取Topic信息的Topic列表,全是通过
this.topicPublishInfoTable.putIfAbsent(topic, new TopicPublishInfo());添加进去的。这刚好也证实了,Producer只关心自己需要的Topic路由信息。
深入到topicRouteData = this.mQClientAPIImpl.getTopicRouteInfoFromNameServer(topic, 1000 * 3);这里就不打算再深入了,具体的网络请求可以参考之前的文章RocketMQ是如何通讯的。我们只需要知道NameSrv返回的是TopicRouteData。
public class TopicRouteData extends RemotingSerializable {
private String orderTopicConf;
private List<QueueData> queueDatas;
private List<BrokerData> brokerDatas;
private HashMap<String/* brokerAddr */, List<String>/* Filter Server */> filterServerTable;
}
但是Producer保存是TopicPublishInfo,如下所示:
public class TopicPublishInfo {
private boolean orderTopic = false;
private boolean haveTopicRouterInfo = false; // 是否有路由信息
private List<MessageQueue> messageQueueList = new ArrayList<MessageQueue>(); // 消息队列信息
private volatile ThreadLocalIndex sendWhichQueue = new ThreadLocalIndex(); // 发送消息负载均衡用
private TopicRouteData topicRouteData;// Topic路由信息
}
因为要考虑到负载均衡,向哪个消息队列发送消息,所以新增了一些属性。Producer拿到返回后,仅仅做一个转换即可。
NameSrv端处理
NameSrv处理实际上是很简单的,直接找到Topic的路由信息填充TopicRouteData返回即可。在NameSrv端,所有的Topic路由信息都存放在RouteInfoManager里面。
public RemotingCommand getRouteInfoByTopic(ChannelHandlerContext ctx, RemotingCommand request) throws RemotingCommandException {
final RemotingCommand response = RemotingCommand.createResponseCommand(null);
final GetRouteInfoRequestHeader requestHeader =
(GetRouteInfoRequestHeader) request.decodeCommandCustomHeader(GetRouteInfoRequestHeader.class);
TopicRouteData topicRouteData = this.namesrvController.getRouteInfoManager().pickupTopicRouteData(requestHeader.getTopic());
if (topicRouteData != null) {
if (this.namesrvController.getNamesrvConfig().isOrderMessageEnable()) {
String orderTopicConf =
this.namesrvController.getKvConfigManager().getKVConfig(NamesrvUtil.NAMESPACE_ORDER_TOPIC_CONFIG,
requestHeader.getTopic());
topicRouteData.setOrderTopicConf(orderTopicConf);
}
byte[] content = topicRouteData.encode();
response.setBody(content);
response.setCode(ResponseCode.SUCCESS);
response.setRemark(null);
return response;
}
response.setCode(ResponseCode.TOPIC_NOT_EXIST);
response.setRemark("No topic route info in name server for the topic: " + requestHeader.getTopic()
+ FAQUrl.suggestTodo(FAQUrl.APPLY_TOPIC_URL));
return response;
}
小结
Producer发送消息时,会在本地查询Topic路由信息,如果没有就新增一个,并立即向NameSrv发起更新该Topic路由信息的请求。此后通过定时任务,定期更新Producer里面所有Topic路由信息。Consumer获取Topic路由信息思路大致与Producer相同,这里就不在赘述了。