1.NameService作用
NameServioce主要作用是为消息生产者和消息消费者提供关于主题Topic的路由信息，NameServioce需存储路由的基本信息，还需要管理Broker节点，包括路由注册和删除等功能
2.路由元信息
RocketMq基于订阅发布机制，一个topic拥有多个消息队列，一个Broker为每个主题默认创建四个读队列和四个写队列。多个Broker组成一个集群，BrokerName由相同的多台Broker组成Master-slave架构，brokerAddrTable map中的对象BrokerData.brokerAddrs属性会存放bokerId的可以，0代表主，大于0代表从。BrokerLiveInfo中会存储lastUpdateTimestamp存储上次收到的Broker心跳包时间
RouteInfoManager部分属性代码：
public class RouteInfoManager {
private static final InternalLogger log = InternalLoggerFactory.getLogger(LoggerName.NAMESRV_LOGGER_NAME);
private final static long BROKER_CHANNEL_EXPIRED_TIME = 1000 * 60 * 2;
//读写锁
private final ReadWriteLock lock = new ReentrantReadWriteLock();
//消息队列路由信息，消息发送时根据路由表进行负载均衡
private final HashMap<String/* topic */, List<QueueData>> topicQueueTable;
//Broker基础信息，包含bokerName、所属集群名称、主备Broker地址
private final HashMap<String/* brokerName */, BrokerData> brokerAddrTable;
//Broker集群信息，在集群中所有的Broker名称
private final HashMap<String/* clusterName */, Set<String/* brokerName */>> clusterAddrTable;
//Broker状态信息，NameService每次收到心跳包时都会替换该信息
private final HashMap<String/* brokerAddr */, BrokerLiveInfo> brokerLiveTable;
//Broker上的FilterService列表，用于类模式消息的过滤
private final HashMap<String/* brokerAddr */, List<String>/* Filter Server */> filterServerTable;
//....
}
3.路由注册
RocketMQ路由注册是通过Broker与NameServer的心跳功能实现。Broker启动时向集群中的所有NameServer发送心跳语句，每隔30s向集群中的所有NameServer发送心跳包。NameServer接收到心跳包会更新brokerLiveTable缓存中的BrokerLiveInfo的lastUpdateTimestamp，然后每隔10s扫描brokerLiveTable，如果连续120s没有心跳包，NamerServer将移除该Broker的路由信息同时关闭Socket链接
-
Broker发送心跳包(BrokerController.start)
this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() { @Override public void run() { try { BrokerController.this.registerBrokerAll(true, false, brokerConfig.isForceRegister()); } catch (Throwable e) { log.error("registerBrokerAll Exception", e); } } }, 1000 * 10, Math.max(10000, Math.min(brokerConfig.getRegisterNameServerPeriod(), 60000)), TimeUnit.MILLISECONDS); /** * public ScheduledFuture<?> scheduleAtFixedRate(Runnable command, * long initialDelay, * long period, * TimeUnit unit); * command：执行线程 * initialDelay：初始化延时 * period：两次开始执行最小间隔时间 * unit：计时单位 */ -
Broker向NamerServer注册 （BrokerOuterAPI#registerBrokerAll）,依次向NameServer发送心跳包
//获取所有的NameServer列表 for (final String namesrvAddr : nameServerAddressList) { brokerOuterExecutor.execute(new Runnable() { @Override public void run() { try { RegisterBrokerResult result = registerBroker(namesrvAddr,oneway, timeoutMills,requestHeader,body); //分别向NameServer注册 if (result != null) { registerBrokerResultList.add(result); } log.info("register broker[{}]to name server {} OK", brokerId, namesrvAddr); } catch (Exception e) { log.warn("registerBroker Exception, {}", namesrvAddr, e); } finally { countDownLatch.countDown(); } } }); }
4.NameServer处理心跳包
网络处理解析类型请求类型为 RequestCode.REGISTER_BROKER,则请求最终转发到RouteInfoManager.registerBroker .路由注册需要加锁，防止并发修改RouteInfoManager中的路由表
集群clusterAddrTable的维护（RouteInfoManager.registerBroker ）
this.lock.writeLock().lockInterruptibly();
Set<String> brokerNames = this.clusterAddrTable.get(clusterName);
if (null == brokerNames) {
brokerNames = new HashSet<String>();
this.clusterAddrTable.put(clusterName, brokerNames);
}
基础信息 brokerAddrTable的维护（RouteInfoManager.registerBroker ）。 首先从brokerAddrTable根据BrokerName尝试获取Broker信息，如果不存在，则创建，如若registerFirst 设置true,则以最新注册信息为准，
BrokerData brokerData = this.brokerAddrTable.get(brokerName);
if (null == brokerData) {
registerFirst = true;
brokerData = new BrokerData(clusterName, brokerName, new HashMap<Long, String>());
this.brokerAddrTable.put(brokerName, brokerData);
}
//省略部分代码
String oldAddr = brokerData.getBrokerAddrs().put(brokerId, brokerAddr);
registerFirst = registerFirst || (null == oldAddr);
若果注册为Master,并且Broker Topic配置信息发生变化或者初次注册，需创建或者更新Topic路由元数据，填充到topicQueueTable,默认主题自动注入路由信息其中包含MixAll.DEFAULT_TOPIC的路由信息。当消息生产者发送主题时，如该主题为创建并且BrokerConfig的autoCreateTopicEnable为true(默认为true),将返回MixAll.DEFAULT_TOPIC的路由信息代码：（RouteInfoManager.createAndUpdateQueueData ）
private void createAndUpdateQueueData(final String brokerName, final TopicConfig topicConfig) {
QueueData queueData = new QueueData();
queueData.setBrokerName(brokerName);
queueData.setWriteQueueNums(topicConfig.getWriteQueueNums());
queueData.setReadQueueNums(topicConfig.getReadQueueNums());
queueData.setPerm(topicConfig.getPerm());
queueData.setTopicSynFlag(topicConfig.getTopicSysFlag());
List<QueueData> queueDataList = this.topicQueueTable.get(topicConfig.getTopicName());
if (null == queueDataList) {
queueDataList = new LinkedList<QueueData>();
queueDataList.add(queueData);
this.topicQueueTable.put(topicConfig.getTopicName(), queueDataList);
log.info("new topic registered, {} {}", topicConfig.getTopicName(), queueData);
} else {
boolean addNewOne = true;
Iterator<QueueData> it = queueDataList.iterator();
while (it.hasNext()) {
QueueData qd = it.next();
if (qd.getBrokerName().equals(brokerName)) {
if (qd.equals(queueData)) {
addNewOne = false;
} else {
log.info("topic changed, {} OLD: {} NEW: {}", topicConfig.getTopicName(), qd,
queueData);
it.remove();
}
}
}
if (addNewOne) {
queueDataList.add(queueData);
}
}
}
BrokerLiveInfo 状态信息维护（RouteInfoManager.registerBroker ），然后是更新BrokerLiveInfo 存活的Broker信息表，Broker执行是否删除的重要依据
BrokerLiveInfo prevBrokerLiveInfo = this.brokerLiveTable.put(brokerAddr,
new BrokerLiveInfo(
System.currentTimeMillis(),
topicConfigWrapper.getDataVersion(),
channel,
haServerAddr));
if (null == prevBrokerLiveInfo) {
log.info("new broker registered, {} HAServer: {}", brokerAddr, haServerAddr);
}
filterServerTable过滤器维护 以后再展开
if (filterServerList != null) {
if (filterServerList.isEmpty()) {
this.filterServerTable.remove(brokerAddr);
} else {
this.filterServerTable.put(brokerAddr, filterServerList);
}
}
5. 路由删除
broker每个30s向NameServer发送一个心跳包，心跳包包含BrojerId,Broker地址，Broker名称，Broker集群名称，Broker关联的FilterServer列表。路由删除存在两处：
- NameServer定时任务每隔十秒扫描brokerLiveTable表，通过lastUpdateTimestamp的时间戳距当前120s,则认为broker失效，一处故障该Broker，关闭时更新（NameServer定时扫描的逻辑在NamesrvConfig启动中 org.apache.rocketmq.namesrv.routeinfo.RouteInfoManager#scanNotActiveBroker）topicQueueTable,brokerAddrTable,brokerLiveTable,filterServerTable（操作数据是会申请读写锁，来防止并发修改）。
- Broker正常被关闭的情况下，回执行unregisterBroker指令（关闭更新内容同上）
删除流程大概分为五步：
1.申请写锁,从brokerAddrTable,brokerLiveTable,filterServerTable移除
2.维护bokerAdderTable,遍历从map中获取的List BrokerData,找到具体Broker从BrokerData中移除，如果移除后list不在包含其他Broker,则删除该brokerAdderTable中该brokerName对应的条目
3.根brokerName，从cluseterAddrTable中找到Broker并从集合中删除。如果移除后集群中不包含任何Broker，则将该集群从集群clusterAddrTable中移除
4.根brokerName遍历所有主题列表，如果队列包含了当前Broker队列，则移除，如果topic只包含待移除的Broker的队列，从路由中删除topic.
5.释放锁，完成路由注册
6.路由发现
RocketMq路由发现是非实时的，当topic路由出现变化后，NameServer不主动推送给客户端，而是由客户端定时拉取主题最新的路由信息。
TopicRouteDate数据结构：
public class TopicRouteData extends RemotingSerializable {
/**
* 顺序消息配置内容，来自于KvConfig
*/
private String orderTopicConf;
/**
* topic队列元数据
*/
private List<QueueData> queueDatas;
/**
* topic分布的broker数据
*/
private List<BrokerData> brokerDatas;
/**
* broker上过滤服务器地址列表
*/
private HashMap<String/* brokerAddr */, List<String>/* Filter Server */> filterServerTable;
NameServer路由发现实现类的实现:org.apache.rocketmq.namesrv.processor.DefaultRequestProcessor#getRouteInfoByTopic
1.调用RouterInfoManager的方法，从路由表topicQueueTable,brokerAdderTable,filterServerTable中分别填充TopicRouteData中的List<‘QueueDate’>,List<‘BrokerData’>和filterServerTable地址表
2.如果找到主题对应的路由信息并且该主题为顺序消费，则从NameServerKVconfig中获取关于顺序消息的相关配置
public RemotingCommand getRouteInfoByTopic(ChannelHandlerContext ctx,
RemotingCommand request) throws RemotingCommandException {
final RemotingCommand response = RemotingCommand.createResponseCommand(null);
final GetRouteInfoRequestHeader requestHeader =
(GetRouteInfoRequestHeader) request.decodeCommandCustomHeader(GetRouteInfoRequestHeader.class);
//构建TopicRouteData
TopicRouteData topicRouteData = this.namesrvController.getRouteInfoManager().pickupTopicRouteData(requestHeader.getTopic());
if (topicRouteData != null) {
//是否开启顺序消息
if (this.namesrvController.getNamesrvConfig().isOrderMessageEnable()) {
String orderTopicConf =
this.namesrvController.getKvConfigManager().getKVConfig(NamesrvUtil.NAMESPACE_ORDER_TOPIC_CONFIG,
requestHeader.getTopic());
topicRouteData.setOrderTopicConf(orderTopicConf);
}
//使用fastjson进行的序列化
byte[] content = topicRouteData.encode();
response.setBody(content);
response.setCode(ResponseCode.SUCCESS);
response.setRemark(null);
return response;
}
//如果为找到 设置TOPIC_NOT_EXIST
response.setCode(ResponseCode.TOPIC_NOT_EXIST);
response.setRemark("No topic route info in name server for the topic: " + requestHeader.getTopic()
+ FAQUrl.suggestTodo(FAQUrl.APPLY_TOPIC_URL));
return response;
}
7.总结
NameServer是整个RocketMQ的中心大脑，NameServer是去中心化的，存在某一时刻NameServer之间数据短暂的不同,但可以保证最终一致, 这恰恰符合mq的设计的初衷,允许短暂的不一致,保证最终一致性。NameServer对路由信息注册，删除使用了读写锁，允许多个消息发送者并发读，保证消息发送时的高并发。同一时刻NameServer只处理一个Broker心跳包，多个心跳包串行执行。
7998
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
