【开源项目】动态线程池框架Hippo4j源码解析

动态线程池框架Hippo4j源码解析

项目简介

Hippo-4J 通过对 JDK 线程池增强，以及扩展三方框架底层线程池等功能，为业务系统提高线上运行保障能力。

快速开始

https://hippo4j.cn/docs/user_docs/user_guide/quick-start/

源码分析

客户端

1. 客户端启动，会创建两个线程池，message-consume和message-produce。(以hippo4j-spring-boot-starter-example 模块为例子)

    @Bean
    @DynamicThreadPool
    public Executor messageConsumeTtlDynamicThreadPool() {
        String threadPoolId = MESSAGE_CONSUME;
        ThreadPoolExecutor customExecutor = ThreadPoolBuilder.builder()
                .dynamicPool()
                .threadFactory(threadPoolId)
                .threadPoolId(threadPoolId)
                .executeTimeOut(800L)
                .waitForTasksToCompleteOnShutdown(true)
                .awaitTerminationMillis(5000L)
                .taskDecorator(new TaskTraceBuilderHandler())
                .build();
        // Ali ttl adaptation use case.
        Executor ttlExecutor = TtlExecutors.getTtlExecutor(customExecutor);
        return ttlExecutor;
    }

    @SpringDynamicThreadPool
    public ThreadPoolExecutor messageProduceDynamicThreadPool() {
        return ThreadPoolBuilder.buildDynamicPoolById(MESSAGE_PRODUCE);
    }

2. DynamicThreadPoolPostProcessor实现了BeanPostProcessor，启动会执行DynamicThreadPoolPostProcessor#postProcessAfterInitialization，会给每个动态线程池加载listener，ClientWorker#addTenantListeners。当出现变更，执行CacheData#safeNotifyListener。

    public Object postProcessAfterInitialization(Object bean, String beanName) throws BeansException {
        if (bean instanceof DynamicThreadPoolExecutor || DynamicThreadPoolAdapterChoose.match(bean)) {
            DynamicThreadPool dynamicThreadPool;
            try {
                dynamicThreadPool = ApplicationContextHolder.findAnnotationOnBean(beanName, DynamicThreadPool.class);
                if (Objects.isNull(dynamicThreadPool)) {
                    // Adapt to lower versions of SpringBoot.
                    dynamicThreadPool = DynamicThreadPoolAnnotationUtil.findAnnotationOnBean(beanName, DynamicThreadPool.class);
                    if (Objects.isNull(dynamicThreadPool)) {
                        return bean;
                    }
                }
            } catch (Exception ex) {
                log.error("Failed to create dynamic thread pool in annotation mode.", ex);
                return bean;
            }
            DynamicThreadPoolExecutor dynamicThreadPoolExecutor;
            if ((dynamicThreadPoolExecutor = DynamicThreadPoolAdapterChoose.unwrap(bean)) == null) {
                dynamicThreadPoolExecutor = (DynamicThreadPoolExecutor) bean;
            }
            DynamicThreadPoolWrapper dynamicThreadPoolWrapper = new DynamicThreadPoolWrapper(dynamicThreadPoolExecutor.getThreadPoolId(), dynamicThreadPoolExecutor);
            ThreadPoolExecutor remoteThreadPoolExecutor = fillPoolAndRegister(dynamicThreadPoolWrapper);
            DynamicThreadPoolAdapterChoose.replace(bean, remoteThreadPoolExecutor);
            subscribeConfig(dynamicThreadPoolWrapper);
            return DynamicThreadPoolAdapterChoose.match(bean) ? bean : remoteThreadPoolExecutor;
        }
        if (bean instanceof DynamicThreadPoolWrapper) {
            DynamicThreadPoolWrapper dynamicThreadPoolWrapper = (DynamicThreadPoolWrapper) bean;
            registerAndSubscribe(dynamicThreadPoolWrapper);
        }
        return bean;
    }

3. DiscoveryClient会进行注册，并且定时发送心跳。

    public DiscoveryClient(HttpAgent httpAgent, InstanceInfo instanceInfo) {
        this.httpAgent = httpAgent;
        this.instanceInfo = instanceInfo;
        this.appPathIdentifier = instanceInfo.getAppName().toUpperCase() + "/" + instanceInfo.getInstanceId();
        this.scheduler = new ScheduledThreadPoolExecutor(
                new Integer(1),
                ThreadFactoryBuilder.builder().daemon(true).prefix("client.discovery.scheduler").build());
        register();
        // Init the schedule tasks.
        initScheduledTasks();
    }

4. 客户端启动的时候，DynamicThreadPoolAutoConfiguration会进行初始化ClientWorker。在IdentifyUtil静态方法块里面执行了DynamicThreadPoolServiceLoader.register(ClientNetworkService.class);，该方法的作用就是根据类名加载SPI，构造实体类存放到DynamicThreadPoolServiceLoader#SERVICES。获取到自定义的网络节点，组装成字符串IDENTIFY。

    static {
        DynamicThreadPoolServiceLoader.register(ClientNetworkService.class);
    }

5. ClientWorker构造方法中，会开启线程。会调用/hippo4j/v1/cs/configs/listener，该方法和apollo的方法如出一辙，都是长轮询。用来监听服务器的修改操作。

executor.schedule(() -> {
            try {
                awaitApplicationComplete.await();
                executorService.execute(new LongPollingRunnable(cacheMap.isEmpty(), cacheCondition));
            } catch (Throwable ex) {
                log.error("Sub check rotate check error.", ex);
            }
        }, 1L, TimeUnit.MILLISECONDS);

6. 当配置变更，客户端会执行CacheData#safeNotifyListener。根据线程池id获取线程池实例，GlobalThreadPoolManage.getExecutorService(threadPoolId).getExecutor()。ServerThreadPoolDynamicRefresh#changePoolInfo用来修改线程池信息，而Hippo4jBaseSendMessageService#sendChangeMessage是用来发送线程池变更的消息的，需要配置。

服务端

1. 服务端处理客户端的注册服务，ApplicationController#addInstance，封装了一个Lease类，保证当前注册实例有效。而清除过期实例是用的EvictionTask。发送定时心跳，主要就是维护Lease对象，续期Lease。

    @Override
    public void register(InstanceInfo registrant) {
        Map<String, Lease<InstanceInfo>> registerMap = registry.get(registrant.getAppName());
        if (registerMap == null) {
            ConcurrentHashMap<String, Lease<InstanceInfo>> registerNewMap = new ConcurrentHashMap<>();
            registerMap = registry.putIfAbsent(registrant.getAppName(), registerNewMap);
            if (registerMap == null) {
                registerMap = registerNewMap;
            }
        }
        Lease<InstanceInfo> existingLease = registerMap.get(registrant.getInstanceId());
        if (existingLease != null && (existingLease.getHolder() != null)) {
            Long existingLastDirtyTimestamp = existingLease.getHolder().getLastDirtyTimestamp();
            Long registrationLastDirtyTimestamp = registrant.getLastDirtyTimestamp();
            if (existingLastDirtyTimestamp > registrationLastDirtyTimestamp) {
                registrant = existingLease.getHolder();
            }
        }
        Lease<InstanceInfo> lease = new Lease<>(registrant);
        if (existingLease != null) {
            lease.setServiceUpTimestamp(existingLease.getServiceUpTimestamp());
        }
        registerMap.put(registrant.getInstanceId(), lease);
        if (InstanceStatus.UP.equals(registrant.getStatus())) {
            lease.serviceUp();
        }
        registrant.setActionType(InstanceInfo.ActionType.ADDED);
        registrant.setLastUpdatedTimestamp();
    }

2. 客户端发送监听配置的请求，服务端会持有长轮询的请求，cLongPollingService#addLongPollingClient，ClientLongPolling是一个Runnable，从请求里面获取clientIdentify，再判断和当前的配置是否一致，不一致则返回最新的数据。

    public void addLongPollingClient(HttpServletRequest req, HttpServletResponse rsp, Map<String, String> clientMd5Map,
                                     int probeRequestSize) {
        String str = req.getHeader(LONG_POLLING_HEADER);
        String noHangUpFlag = req.getHeader(LONG_POLLING_NO_HANG_UP_HEADER);
        int delayTime = SwitchService.getSwitchInteger(SwitchService.FIXED_DELAY_TIME, 500);
        long timeout = Math.max(10000, Long.parseLong(str) - delayTime);
        if (isFixedPolling()) {
            timeout = Math.max(10000, getFixedPollingInterval());
        } else {
            List<String> changedGroups = Md5ConfigUtil.compareMd5(req, clientMd5Map);
            if (!changedGroups.isEmpty()) {
                generateResponse(rsp, changedGroups);
                return;
            } else if (noHangUpFlag != null && noHangUpFlag.equalsIgnoreCase(TRUE_STR)) {
                log.info("New initializing cacheData added in.");
                return;
            }
        }
        String clientIdentify = RequestUtil.getClientIdentify(req);
        final AsyncContext asyncContext = req.startAsync();
        asyncContext.setTimeout(0L);
        ConfigExecutor.executeLongPolling(new ClientLongPolling(asyncContext, clientMd5Map, clientIdentify, probeRequestSize,
                timeout - delayTime, Pair.of(req.getHeader(CLIENT_APP_NAME_HEADER), req.getHeader(CLIENT_VERSION))));
    }

总结

1. 服务端类似于注册中心（eureka），会将客户端的信息展示出来；同样，服务端也是一个配置中心（nacos），当服务端的配置有修改，客户端会监听到服务端的配置变更，对线程池的改变实时生效。