RocketMQ源码初识一 NameServer源码

官方doc中描述

名称服务充当路由消息的提供者。生产者或消费者能够通过名字服务查找各主题相应的Broker IP列表。多个Namesrv实例组成集群，但相互独立，没有信息交换。

NameServer：NameServer是一个非常简单的Topic路由注册中心，其角色类似Dubbo中的zookeeper，支持Broker的动态注册与发现。主要包括两个功能：Broker管理，NameServer接受Broker集群的注册信息并且保存下来作为路由信息的基本数据。然后提供心跳检测机制，检查Broker是否还存活；路由信息管理，每个NameServer将保存关于Broker集群的整个路由信息和用于客户端查询的队列信息。然后Producer和Conumser通过NameServer就可以知道整个Broker集群的路由信息，从而进行消息的投递和消费。NameServer通常也是集群的方式部署，各实例间相互不进行信息通讯。Broker是向每一台NameServer注册自己的路由信息，所以每一个NameServer实例上面都保存一份完整的路由信息。当某个NameServer因某种原因下线了，Broker仍然可以向其它NameServer同步其路由信息，Producer,Consumer仍然可以动态感知Broker的路由的信息。

nameServer大概用到的模块

源码总结

NameServer代码很少，启动时会主要获取配置，设置配置，然后初始化缓存对象，再调用remoting模块创建一个romotingServer，并设置处理请求类为DefaultRequestProcessor

启动romotingServer，会启动一个netty服务，每次接收到请求会交给DefaultRequestProcessor

代码流程图

源代码

mai方法

NameServer启动入口，实际有用的功能就两行，一个是创建controller，一个是启动controller

public static void main(String[] args) {
    main0(args);
}

public static NamesrvController main0(String[] args) {
    try {
    	//创建controller
        NamesrvController controller = createNamesrvController(args);
        //启动controller
        start(controller);
        String tip = "The Name Server boot success. serializeType=" + RemotingCommand.getSerializeTypeConfigInThisServer();
        log.info(tip);
        System.out.printf("%s%n", tip);
        return controller;
    } catch (Throwable e) {
        e.printStackTrace();
        System.exit(-1);
    }
    return null;
}

createNamesrvController

获取启动参数，判断是否要启动，根据参数中的配置，重写配置类，并根据配置类创建NamesrvController对象

public static NamesrvController createNamesrvController(String[] args) throws IOException, JoranException {
    //设置当前MQ的版本
    System.setProperty(RemotingCommand.REMOTING_VERSION_KEY, Integer.toString(MQVersion.CURRENT_VERSION));
    //构建命令行工具，用来存储支持的参数，和参数的含义
    // 添加 h->help 与n->namesrvAddr
    Options options = ServerUtil.buildCommandlineOptions(new Options());
    // 添加 c->configFile 与 p-> printConfigItem
    commandLine = ServerUtil.parseCmdLine("mqnamesrv", args, buildCommandlineOptions(options), new PosixParser());
    if (null == commandLine) {
        System.exit(-1);
        return null;
    }
    //构建NameServer的配置信息
    final NamesrvConfig namesrvConfig = new NamesrvConfig();
    //构建netty的配置信息
    final NettyServerConfig nettyServerConfig = new NettyServerConfig();
    //netty监听端口 默认9876(-c指定的配置文件中可修改)
    nettyServerConfig.setListenPort(9876);
    //如果启动的时候，传了
    if (commandLine.hasOption('c')) {
        //如果用户传了-c 指定了配置文件路径
        String file = commandLine.getOptionValue('c');
        if (file != null) {
            //加载配置文件
            InputStream in = new BufferedInputStream(new FileInputStream(file));
            properties = new Properties();
            properties.load(in);
            //覆盖属性
            MixAll.properties2Object(properties, namesrvConfig);
            //覆盖netty相关属性
            MixAll.properties2Object(properties, nettyServerConfig);
            namesrvConfig.setConfigStorePath(file);
            System.out.printf("load config properties file OK, %s%n", file);
            in.close();
        }
    }
    //如果传-p 打印所有配置，并退出程序
    if (commandLine.hasOption('p')) {
        InternalLogger console = InternalLoggerFactory.getLogger(LoggerName.NAMESRV_CONSOLE_NAME);
        MixAll.printObjectProperties(console, namesrvConfig);
        MixAll.printObjectProperties(console, nettyServerConfig);
        System.exit(0);
    }

    MixAll.properties2Object(ServerUtil.commandLine2Properties(commandLine), namesrvConfig);
    //环境变量中是否有RocketmqHome属性，没有的话，报错退出
    if (null == namesrvConfig.getRocketmqHome()) {
        System.out.printf("Please set the %s variable in your environment to match the location of the RocketMQ installation%n", MixAll.ROCKETMQ_HOME_ENV);
        System.exit(-2);
    }
    LoggerContext lc = (LoggerContext) LoggerFactory.getILoggerFactory();
    JoranConfigurator configurator = new JoranConfigurator();
    configurator.setContext(lc);
    lc.reset();
    //加载日志配置
    configurator.doConfigure(namesrvConfig.getRocketmqHome() + "/conf/logback_namesrv.xml");
    log = InternalLoggerFactory.getLogger(LoggerName.NAMESRV_LOGGER_NAME);
    MixAll.printObjectProperties(log, namesrvConfig);
    MixAll.printObjectProperties(log, nettyServerConfig);
    //创建Controller，将配置对象传进去
    final NamesrvController controller = new NamesrvController(namesrvConfig, nettyServerConfig);
    // remember all configs to prevent discard
    controller.getConfiguration().registerConfig(properties);
    return controller;
}

Options与PosixParser

org.apache.commons.cli下提供的工具类，可自定义命令行参数，可添加支持的命令，可根据输入的命令执行对应的操作

在这里定义了namerServer启动时支持的参数，可根据传入的参数，执行不同的操作

查看执行效果：
1、idea启动时候，在Program arguments中传-h、-p、-c、-n
2、控制台切目录${ROCKETMQ_HOME}/lib下执行

java -Djava.ext.dirs=../lib -cp rocketmq-namesrv-4.7.1.jar org.apache.rocketmq.namesrv.NamesrvStartup -h

传-h的执行结果

usage: mqnamesrv [-c <arg>] [-h] [-n <arg>] [-p]
 -c,--configFile <arg>    Name server config properties file
 -h,--help                Print help
 -n,--namesrvAddr <arg>   Name server address list, eg: 192.168.0.1:9876;192.168.0.2:9876
 -p,--printConfigItem     Print all config item

传-p的执行结果

19:19:35.130 [main] INFO  RocketmqNamesrvConsole - rocketmqHome=
19:19:35.131 [main] INFO  RocketmqNamesrvConsole - kvConfigPath=/root/namesrv/kvConfig.json
19:19:35.132 [main] INFO  RocketmqNamesrvConsole - configStorePath=/root/namesrv/namesrv.properties
19:19:35.132 [main] INFO  RocketmqNamesrvConsole - productEnvName=center
19:19:35.132 [main] INFO  RocketmqNamesrvConsole - clusterTest=false
19:19:35.132 [main] INFO  RocketmqNamesrvConsole - orderMessageEnable=false
19:19:35.132 [main] INFO  RocketmqNamesrvConsole - listenPort=9876
19:19:35.132 [main] INFO  RocketmqNamesrvConsole - serverWorkerThreads=8
19:19:35.132 [main] INFO  RocketmqNamesrvConsole - serverCallbackExecutorThreads=0
19:19:35.132 [main] INFO  RocketmqNamesrvConsole - serverSelectorThreads=3
19:19:35.132 [main] INFO  RocketmqNamesrvConsole - serverOnewaySemaphoreValue=256
19:19:35.132 [main] INFO  RocketmqNamesrvConsole - serverAsyncSemaphoreValue=64
19:19:35.132 [main] INFO  RocketmqNamesrvConsole - serverChannelMaxIdleTimeSeconds=120
19:19:35.132 [main] INFO  RocketmqNamesrvConsole - serverSocketSndBufSize=65535
19:19:35.132 [main] INFO  RocketmqNamesrvConsole - serverSocketRcvBufSize=65535
19:19:35.132 [main] INFO  RocketmqNamesrvConsole - serverPooledByteBufAllocatorEnable=true
19:19:35.132 [main] INFO  RocketmqNamesrvConsole - useEpollNativeSelector=false

NamesrvController构造函数

初始化信息，其中最重要的就是RouteInfoManager

public NamesrvController(NamesrvConfig namesrvConfig, NettyServerConfig nettyServerConfig) {
    //naerServer的配置
    this.namesrvConfig = namesrvConfig;
    //netty的配置
    this.nettyServerConfig = nettyServerConfig;
    //kv配置管理
    this.kvConfigManager = new KVConfigManager(this);
    //路由信息管理
    this.routeInfoManager = new RouteInfoManager();
    //检测broker下线服务
    this.brokerHousekeepingService = new BrokerHousekeepingService(this);
    this.configuration = new Configuration(
        log,
        this.namesrvConfig, this.nettyServerConfig
    );
    this.configuration.setStorePathFromConfig(this.namesrvConfig, "configStorePath");
}

RouteInfoManager构造函数

初始化hashMap

private final HashMap<String/* topic */, List<QueueData>> topicQueueTable;
private final HashMap<String/* brokerName */, BrokerData> brokerAddrTable;
private final HashMap<String/* clusterName */, Set<String/* brokerName */>> clusterAddrTable;
private final HashMap<String/* brokerAddr */, BrokerLiveInfo> brokerLiveTable;
private final HashMap<String/* brokerAddr */, List<String>/* Filter Server */> filterServerTable;

public RouteInfoManager() {
    //记录每个topic的配置信息
    this.topicQueueTable = new HashMap<String, List<QueueData>>(1024);
    //记录每个broker的地址，和broker所属集群
    this.brokerAddrTable = new HashMap<String, BrokerData>(128);
    //记录集群名字对应的broker集合
    this.clusterAddrTable = new HashMap<String, Set<String>>(32);
    //记录在线的broker
    this.brokerLiveTable = new HashMap<String, BrokerLiveInfo>(256);
    this.filterServerTable = new HashMap<String, List<String>>(256);
}

start(controller);

public static NamesrvController start(final NamesrvController controller) throws Exception {
    if (null == controller) {
        throw new IllegalArgumentException("NamesrvController is null");
    }
    //初始化
    boolean initResult = controller.initialize();
    if (!initResult) {
        controller.shutdown();
        System.exit(-3);
    }
    //监听关闭钩子
    Runtime.getRuntime().addShutdownHook(new ShutdownHookThread(log, new Callable<Void>() {
        @Override
        public Void call() throws Exception {
            controller.shutdown();
            return null;
        }
    }));
    //启动
    controller.start();
    return controller;
}

初始化controller.initialize

1、创建NettyRemotingServer：用来接收远端请求
2、创建线程池
3、注册处理器，根据不同请求类型，调用对应操作
4、创建定时任务，定期检查过期broker

public boolean initialize() {
    // 从文件中加载kv配置信息
    this.kvConfigManager.load();
    // 初始化netty服务（接收处理远端请求）
    this.remotingServer = new NettyRemotingServer(this.nettyServerConfig, this.brokerHousekeepingService);
    // 处理远程请求线程池
    this.remotingExecutor = Executors.newFixedThreadPool(nettyServerConfig.getServerWorkerThreads(),
        new ThreadFactoryImpl("RemotingExecutorThread_"));
    // 设置netty服务器处理请求的类是DefaultRequestProcessor
    this.registerProcessor();
    // 每隔10秒检查掉线的broker
    this.scheduledExecutorService.scheduleAtFixedRate(NamesrvController.this.routeInfoManager::scanNotActiveBroker,
        5, 10, TimeUnit.SECONDS);
    // 每隔10分钟输出kv配置信息
    this.scheduledExecutorService.scheduleAtFixedRate(NamesrvController.this.kvConfigManager::printAllPeriodically,
        1, 10, TimeUnit.MINUTES);
    // ssl相关配置，此处可以忽略
    if (TlsSystemConfig.tlsMode != TlsMode.DISABLED) {
        ...
    }
    return true;
}

DefaultRequestProcessor

设置netty实际处理请求时，使用DefaultRequestProcessor处理

private void registerProcessor() {
    if (namesrvConfig.isClusterTest()) {

        this.remotingServer.registerDefaultProcessor(
            new ClusterTestRequestProcessor(this, namesrvConfig.getProductEnvName()), this.remotingExecutor);
    } else {
        //处理远端的服务中注册处理类是DefaultRequestProcessor
        this.remotingServer.registerDefaultProcessor(new DefaultRequestProcessor(this), this.remotingExecutor);
    }
}

NettyRemotingServer构造函数（remoting模块）

初始化netty服务相关类

public NettyRemotingServer(final NettyServerConfig nettyServerConfig,
    final ChannelEventListener channelEventListener) {
    super(nettyServerConfig.getServerOnewaySemaphoreValue(), nettyServerConfig.getServerAsyncSemaphoreValue());
    this.serverBootstrap = new ServerBootstrap();
    this.nettyServerConfig = nettyServerConfig;
    //监听器
    this.channelEventListener = channelEventListener;
    int publicThreadNums = nettyServerConfig.getServerCallbackExecutorThreads();
    if (publicThreadNums <= 0) {
        publicThreadNums = 4;
    }
    //发布事件用的线程池
    this.publicExecutor = Executors.newFixedThreadPool(publicThreadNums, new ThreadFactory() {
        private AtomicInteger threadIndex = new AtomicInteger(0);
        @Override
        public Thread newThread(Runnable r) {
            return new Thread(r, "NettyServerPublicExecutor_" + this.threadIndex.incrementAndGet());
        }
    });
    if (useEpoll()) {
        this.eventLoopGroupBoss = new EpollEventLoopGroup(1, new ThreadFactory() {
            private AtomicInteger threadIndex = new AtomicInteger(0);

            @Override
            public Thread newThread(Runnable r) {
                return new Thread(r, String.format("NettyEPOLLBoss_%d", this.threadIndex.incrementAndGet()));
            }
        });

        this.eventLoopGroupSelector =
            new EpollEventLoopGroup(nettyServerConfig.getServerSelectorThreads(), new ThreadFactory() {
                private AtomicInteger threadIndex = new AtomicInteger(0);
                private int threadTotal = nettyServerConfig.getServerSelectorThreads();

                @Override
                public Thread newThread(Runnable r) {
                    return new Thread(r, String.format("NettyServerEPOLLSelector_%d_%d", threadTotal,
                        this.threadIndex.incrementAndGet()));
                }
            });
    } else {
        this.eventLoopGroupBoss = new NioEventLoopGroup(1, new ThreadFactory() {
            private AtomicInteger threadIndex = new AtomicInteger(0);

            @Override
            public Thread newThread(Runnable r) {
                return new Thread(r, String.format("NettyNIOBoss_%d", this.threadIndex.incrementAndGet()));
            }
        });
        this.eventLoopGroupSelector =
            new NioEventLoopGroup(nettyServerConfig.getServerSelectorThreads(), new ThreadFactory() {
                private AtomicInteger threadIndex = new AtomicInteger(0);
                private int threadTotal = nettyServerConfig.getServerSelectorThreads();

                @Override
                public Thread newThread(Runnable r) {
                    return new Thread(r, String.format("NettyServerNIOSelector_%d_%d", threadTotal,
                        this.threadIndex.incrementAndGet()));
                }
            });
    }
    loadSslContext();
}

scanNotActiveBroker 检查不是活跃的broker

判断在线的broker是否2分钟没有收到更新信息，如果是移除相关信息

public void scanNotActiveBroker() {
    //获取所有在线的broker
    Iterator<Entry<String, BrokerLiveInfo>> it = this.brokerLiveTable.entrySet().iterator();
    while (it.hasNext()) {
        Entry<String, BrokerLiveInfo> next = it.next();
        //获取最后一次更新事件
        long last = next.getValue().getLastUpdateTimestamp();
        //如果超过2分钟没有更新
        if ((last + BROKER_CHANNEL_EXPIRED_TIME) < System.currentTimeMillis()) {
            //关闭管道
            RemotingUtil.closeChannel(next.getValue().getChannel());
            //从live中移除
            it.remove();
            log.warn("The broker channel expired, {} {}ms", next.getKey(), BROKER_CHANNEL_EXPIRED_TIME);
            //移除相关信息
            this.onChannelDestroy(next.getKey(), next.getValue().getChannel());
        }
    }
}

controller.start() 启动controller

public void start() throws Exception {
     //启动netty服务
     this.remotingServer.start();
     if (this.fileWatchService != null) {
         this.fileWatchService.start();
     }
 }

remotingServer.start 启动netty服务

启动netty服务并绑定端口，设置netty相关的处理类

public void start() {
    // netty
    this.defaultEventExecutorGroup =
        new DefaultEventExecutorGroup(nettyServerConfig.getServerWorkerThreads(), new ThreadFactory() {
            private AtomicInteger threadIndex = new AtomicInteger(0);

            @Override
            public Thread newThread(Runnable r) {
                return new Thread(r, "NettyServerCodecThread_" + this.threadIndex.incrementAndGet());
            }
        });
    // 前期准备工作，初始化相关类
    prepareSharableHandlers();
    // 构建netty服务
    ServerBootstrap childHandler = this.serverBootstrap.group(this.eventLoopGroupBoss, this.eventLoopGroupSelector)
        .channel(useEpoll() ? EpollServerSocketChannel.class : NioServerSocketChannel.class)
        .option(ChannelOption.SO_BACKLOG, 1024).option(ChannelOption.SO_REUSEADDR, true)
        .option(ChannelOption.SO_KEEPALIVE, false).childOption(ChannelOption.TCP_NODELAY, true)
        .childOption(ChannelOption.SO_SNDBUF, nettyServerConfig.getServerSocketSndBufSize())
        .childOption(ChannelOption.SO_RCVBUF, nettyServerConfig.getServerSocketRcvBufSize())
        //设置端口，默认是9876，可从配置中指定
        .localAddress(new InetSocketAddress(this.nettyServerConfig.getListenPort()))
        .childHandler(new ChannelInitializer<SocketChannel>() {
            @Override
            public void initChannel(SocketChannel ch) throws Exception {
                //处理握手信息（权限和ssl）
                ch.pipeline().addLast(defaultEventExecutorGroup, HANDSHAKE_HANDLER_NAME, handshakeHandler);
                //设置编码和解码
                ch.pipeline().addLast(defaultEventExecutorGroup, encoder, new NettyDecoder(),
                    new IdleStateHandler(0, 0, nettyServerConfig.getServerChannelMaxIdleTimeSeconds()),
                    //连接管理，有连接发生变化时，发送事件
                    connectionManageHandler,
                    //具体处理接收到的请求
                    serverHandler);
            }
        });

    if (nettyServerConfig.isServerPooledByteBufAllocatorEnable()) {
        childHandler.childOption(ChannelOption.ALLOCATOR, PooledByteBufAllocator.DEFAULT);
    }
    try {
        //绑定端口
        ChannelFuture sync = this.serverBootstrap.bind().sync();
        InetSocketAddress addr = (InetSocketAddress)sync.channel().localAddress();
        this.port = addr.getPort();
    } catch (InterruptedException e1) {
        throw new RuntimeException("this.serverBootstrap.bind().sync() InterruptedException", e1);
    }

    if (this.channelEventListener != null) {
        this.nettyEventExecutor.start();
    }
    this.timer.scheduleAtFixedRate(new TimerTask() {
        @Override
        public void run() {
            try {
                NettyRemotingServer.this.scanResponseTable();
            } catch (Throwable e) {
                log.error("scanResponseTable exception", e);
            }
        }
    }, 1000 * 3, 1000);
}

netty接收请求

connectionManageHandler连接管理

连接有状态变化时，发布相应事件

serverHandler 实际处理请求

netty接收到消息

class NettyServerHandler extends SimpleChannelInboundHandler<RemotingCommand> {

    @Override
    protected void channelRead0(ChannelHandlerContext ctx, RemotingCommand msg) throws Exception {
        //接收到消息时触发
        processMessageReceived(ctx, msg);
    }
}

接收到消息，根据类型调用

    public void processMessageReceived(ChannelHandlerContext ctx, RemotingCommand msg) throws Exception {
        final RemotingCommand cmd = msg;
        if (cmd != null) {
            switch (cmd.getType()) {
                case REQUEST_COMMAND:
                    processRequestCommand(ctx, cmd);
                    break;
                case RESPONSE_COMMAND:
                    processResponseCommand(ctx, cmd);
                    break;
                default:
                    break;
            }
        }
    }

processRequestCommand 处理request类型请求

摘取方法里部分代码，processor为初始化controller时设置的DefaultRequestProcessor

if (pair.getObject1() instanceof AsyncNettyRequestProcessor) {
    //异步请求
    AsyncNettyRequestProcessor processor = (AsyncNettyRequestProcessor)pair.getObject1();
    processor.asyncProcessRequest(ctx, cmd, callback);
} else {
    //同步请求
    NettyRequestProcessor processor = pair.getObject1();
    RemotingCommand response = processor.processRequest(ctx, cmd);
    callback.callback(response);
}

DefaultRequestProcessor.processRequest

根据不同的请求方式，返回不同的数据
例如：注册broker，获取topic等等

public RemotingCommand processRequest(ChannelHandlerContext ctx,
    RemotingCommand request) throws RemotingCommandException {
    if (ctx != null) {
        //打印日志
        log.debug("receive request, {} {} {}",
            request.getCode(),
            RemotingHelper.parseChannelRemoteAddr(ctx.channel()),
            request);
    }
    //根据请求类型，做对应操作
    switch (request.getCode()) {
        //添加kv配置
        case RequestCode.PUT_KV_CONFIG:
            return this.putKVConfig(ctx, request);
        //获取kv配置
        case RequestCode.GET_KV_CONFIG:
            return this.getKVConfig(ctx, request);
        //删除kv配置
        case RequestCode.DELETE_KV_CONFIG:
            return this.deleteKVConfig(ctx, request);
        //查询broker版本（心跳，更新broker最后更新时间）
        case RequestCode.QUERY_DATA_VERSION:
            return queryBrokerTopicConfig(ctx, request);
        //注册broker
        case RequestCode.REGISTER_BROKER:
            Version brokerVersion = MQVersion.value2Version(request.getVersion());
            if (brokerVersion.ordinal() >= MQVersion.Version.V3_0_11.ordinal()) {
                return this.registerBrokerWithFilterServer(ctx, request);
            } else {
                return this.registerBroker(ctx, request);
            }
        //取消broker
        case RequestCode.UNREGISTER_BROKER:
            return this.unregisterBroker(ctx, request);
        //根据topic获取topic信息
        case RequestCode.GET_ROUTEINFO_BY_TOPIC:
            return this.getRouteInfoByTopic(ctx, request);
        //获取所有broker和所有集群信息
        case RequestCode.GET_BROKER_CLUSTER_INFO:
            return this.getBrokerClusterInfo(ctx, request);
        case RequestCode.WIPE_WRITE_PERM_OF_BROKER:
            return this.wipeWritePermOfBroker(ctx, request);
        //获取所有topic
        case RequestCode.GET_ALL_TOPIC_LIST_FROM_NAMESERVER:
            return getAllTopicListFromNameserver(ctx, request);
        //删除topic
        case RequestCode.DELETE_TOPIC_IN_NAMESRV:
            return deleteTopicInNamesrv(ctx, request);
        //获取namespace下所有kv
        case RequestCode.GET_KVLIST_BY_NAMESPACE:
            return this.getKVListByNamespace(ctx, request);
        //获取集群下的topic(先获取集群下的broker，然后获取broker下的topic)
        case RequestCode.GET_TOPICS_BY_CLUSTER:
            return this.getTopicsByCluster(ctx, request);
        //获取所有系统的topic
        case RequestCode.GET_SYSTEM_TOPIC_LIST_FROM_NS:
            return this.getSystemTopicListFromNs(ctx, request);
        case RequestCode.GET_UNIT_TOPIC_LIST:
            return this.getUnitTopicList(ctx, request);
        case RequestCode.GET_HAS_UNIT_SUB_TOPIC_LIST:
            return this.getHasUnitSubTopicList(ctx, request);
        case RequestCode.GET_HAS_UNIT_SUB_UNUNIT_TOPIC_LIST:
            return this.getHasUnitSubUnUnitTopicList(ctx, request);
        //更新nameServer配置
        case RequestCode.UPDATE_NAMESRV_CONFIG:
            return this.updateConfig(ctx, request);
        //获取nameServer配置
        case RequestCode.GET_NAMESRV_CONFIG:
            return this.getConfig(ctx, request);
        default:
            break;
    }
    return null;
}