Nacos（1.4.2）注册中心原理及源码系列（三）- 如何支持高并发注册

如何支持高并发注册（异步任务与内存队列设计原理及源码剖析）

    之前主要分析了Spring Cloud集成Nacos client的服务注册和服务拉取的逻辑，现在接着分析一下Nacos Server注册中心的核心功能逻辑及源码，首先来分析Nacos怎么能支持高并发的Intance的注册的。

    先直接给答案：采用内存队列的方式进行服务注册

    也就是说客户端在把自己的信息注册到Nacos Server的时候，并不是同步把信息写入到注册表中的，而且采取了先写入内存队列中，然后用独立的线程池来消费队列进行注册的。

源码如下：

// com.alibaba.nacos.naming.controllers.InstanceController#register
@CanDistro
@PostMapping
@Secured(parser = NamingResourceParser.class, action = ActionTypes.WRITE)
public String register(HttpServletRequest request) throws Exception {

 	// 省略其它代码。。。。。。。
    
    serviceManager.registerInstance(namespaceId, serviceName, instance);
    return "ok";
}
// com.alibaba.nacos.naming.core.ServiceManager#addInstance
public void addInstance(String namespaceId, String serviceName, boolean ephemeral, Instance... ips)
    throws NacosException {
    // com.alibaba.nacos.naming.iplist.ephemeral.public##DEFAULT_GROUP@@nacos-demo
    String key = KeyBuilder.buildInstanceListKey(namespaceId, serviceName, ephemeral);
    Service service = getService(namespaceId, serviceName);
    synchronized (service) {
        // 把需要添加的Instance加入到注册表的副本中，然后返回一个添加好instance的instance list
        List<Instance> instanceList = addIpAddresses(service, ephemeral, ips);

        Instances instances = new Instances();
        instances.setInstanceList(instanceList);
		
        consistencyService.put(key, instances);
    }
}
// com.alibaba.nacos.naming.consistency.DelegateConsistencyServiceImpl#put
@Override
public void put(String key, Record value) throws NacosException {
    mapConsistencyService(key).put(key, value);
}

// com.alibaba.nacos.naming.consistency.DelegateConsistencyServiceImpl#mapConsistencyService
private ConsistencyService mapConsistencyService(String key) {
    // com.alibaba.nacos.naming.iplist.ephemeral.public##DEFAULT_GROUP@@nacos-demo
    // 判断是否有 INSTANCE_LIST_KEY_PREFIX + EPHEMERAL_KEY_PREFIX
    // 实际是判断在key中是否存在ephemeral，如果存在证明是一个临时实例，那么就返回临时实例的service，反之返回持久实例的service
    return KeyBuilder.matchEphemeralKey(key) ? ephemeralConsistencyService : persistentConsistencyService;
}

// com.alibaba.nacos.naming.consistency.ephemeral.distro.DistroConsistencyServiceImpl#put
@Override
public void put(String key, Record value) throws NacosException {
    onPut(key, value);
    distroProtocol.sync(new DistroKey(key, KeyBuilder.INSTANCE_LIST_KEY_PREFIX), DataOperation.CHANGE,
                        globalConfig.getTaskDispatchPeriod() / 2);
}

// com.alibaba.nacos.naming.consistency.ephemeral.distro.DistroConsistencyServiceImpl#onPut
public void onPut(String key, Record value) {
    if (KeyBuilder.matchEphemeralInstanceListKey(key)) {
        Datum<Instances> datum = new Datum<>();
        datum.value = (Instances) value;
        datum.key = key;
        datum.timestamp.incrementAndGet();
        // 把Instance数据放入dataStore中
        dataStore.put(key, datum);
    }

    if (!listeners.containsKey(key)) {
        return;
    }
    notifier.addTask(key, DataOperation.CHANGE);
}
// com.alibaba.nacos.naming.consistency.ephemeral.distro.DistroConsistencyServiceImpl.Notifier#addTask
public void addTask(String datumKey, DataOperation action) {

    if (services.containsKey(datumKey) && action == DataOperation.CHANGE) {
        return;
    }
    if (action == DataOperation.CHANGE) {
        services.put(datumKey, StringUtils.EMPTY);
    }
    // key 和 DataOperation 放入队列中，队列的数据结构为一个ArrayBlockingQueue，定义如下
    // private BlockingQueue<Pair<String, DataOperation>> tasks = new ArrayBlockingQueue<>(1024 * 1024);
    tasks.offer(Pair.with(datumKey, action));
}

// com.alibaba.nacos.naming.consistency.ephemeral.distro.DistroConsistencyServiceImpl.Notifier#run
@Override
public void run() {
    Loggers.DISTRO.info("distro notifier started");
    for (; ; ) {
        try {
            // 从队列中取出
            Pair<String, DataOperation> pair = tasks.take();
            handle(pair);
        } catch (Throwable e) {
            Loggers.DISTRO.error("[NACOS-DISTRO] Error while handling notifying task", e);
        }
    }
}
// com.alibaba.nacos.naming.consistency.ephemeral.distro.DistroConsistencyServiceImpl.Notifier#handle
private void handle(Pair<String, DataOperation> pair) {
    try {
        String datumKey = pair.getValue0();
        DataOperation action = pair.getValue1();
        services.remove(datumKey);
        int count = 0;
        if (!listeners.containsKey(datumKey)) {
            return;
        }
        for (RecordListener listener : listeners.get(datumKey)) {
            count++;
            try {
                if (action == DataOperation.CHANGE) {
                    // 根据key把instances从dataStore中取出，并调用change方法
                    listener.onChange(datumKey, dataStore.get(datumKey).value);
                    continue;
                }

                if (action == DataOperation.DELETE) {
                    // 调用onDelete方法
                    listener.onDelete(datumKey);
                    continue;
                }
            } catch (Throwable e) {
                Loggers.DISTRO.error("[NACOS-DISTRO] error while notifying listener of key: {}", datumKey, e);
            }
        }
        if (Loggers.DISTRO.isDebugEnabled()) {
            Loggers.DISTRO
                .debug("[NACOS-DISTRO] datum change notified, key: {}, listener count: {}, action: {}",
                       datumKey, count, action.name());
        }
    } catch (Throwable e) {
        Loggers.DISTRO.error("[NACOS-DISTRO] Error while handling notifying task", e);
    }
}

    从源码可看出最终会执行listener.onChange()这个方法，并把Instances传入，然后进行真正的注册逻辑，这里的设计就是为了提高Nacos Server的并发注册量，如果你非常关注Nacos性能相关问题，可以查看官方的压测报告（Nacos服务发现性能测试报告），或者自己去做一下压测。

    这里再提一下，在进行队列消费的时候其实最终也是采用的JDK的线程池，追踪到实例化线程线程池的代码为：

// com.alibaba.nacos.common.executor.ExecutorFactory.Managed#newSingleScheduledExecutorService
public static ScheduledExecutorService newSingleScheduledExecutorService(final String group,
                                                                         final ThreadFactory threadFactory) {
    // 其实这里使用的是JDK里的ScheduledExecutorService
    ScheduledExecutorService executorService = Executors.newScheduledThreadPool(1, threadFactory);
    THREAD_POOL_MANAGER.register(DEFAULT_NAMESPACE, group, executorService);
    return executorService;
}