美团动态ThreadPoolExecutor底层实现源码实战

玛卡巴咖

已于 2023-10-25 11:17:23 修改

阅读量366

点赞数

文章标签： 1024程序员节

于 2023-10-24 21:09:39 首次发布

本文链接：https://blog.csdn.net/qq_67801847/article/details/134021089

版权

开篇：介绍springboot连接nacos实现动态线程池，同时得安装nacos，同时代码将有两个模块，dtp-spring-boot-starter 与 user 模块，前者将是独立的动态线程池，可以引入自己的项目中，后者模块主要用于测试与使用动态线程池模块。

依赖与工具描述
springboot 2.3.9.RELEASE
nacos-config-spring-boot-starter 0.2.10
nacos 2.1.1

注意springboot与nacos的适配版本！

完整代码gitee：ThreadPopl-starter: 动态线程池代码

一，搭建实现

1.创建两个模块并配置

user：测试模块（夫模块）

Maven依赖：

    <parent>
        <artifactId>spring-boot-starter-parent</artifactId>
        <groupId>org.springframework.boot</groupId>
        <version>2.3.9.RELEASE</version>
<!--        <version>2.0.3.RELEASE</version>-->
    </parent>

    <groupId>org.example</groupId>
    <artifactId>ThreadPool-demo</artifactId>
    <packaging>pom</packaging>
    <version>1.0-SNAPSHOT</version>

    <modules>
        <module>dtp-spring-boot-starter</module>
    </modules>

    <properties>
        <maven.compiler.source>8</maven.compiler.source>
        <maven.compiler.target>8</maven.compiler.target>
    </properties>

在配置文件中写入连接nacos的配置代码

nacos:
  config:
    server-addr: 192.168.1.116:8848
    username: nacos
    password: nacos
    #命名空间不要写错
    namespace: 6b5d4d2a-5385-4d9f-85a1-18b748b8256c

dtp-spring-boot-starter：动态线程池（子模块）

Maven依赖：

    <parent>
        <artifactId>user</artifactId>
        <groupId>org.example</groupId>
        <version>1.0-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>dtp-spring-boot-starter</artifactId>

    <properties>
        <maven.compiler.source>8</maven.compiler.source>
        <maven.compiler.target>8</maven.compiler.target>
    </properties>

    <dependencies>
        <dependency>
            <groupId>com.alibaba.boot</groupId>
            <artifactId>nacos-config-spring-boot-starter</artifactId>
            <version>0.2.10</version>
        </dependency>
<!--        <dependency>-->
<!--            <groupId>com.alibaba.boot</groupId>-->
<!--            <artifactId>nacos-config-spring-boot-actuator</artifactId>-->
<!--            <version>0.2.10</version>-->
<!--        </dependency>-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-configuration-processor</artifactId>
        </dependency>

    </dependencies>

2. 开始编写动态线程池配置（dtp-spring-boot-starter模块）

创建动态线程池对象

package com.laoyang.dtp;

import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;

/**
 * @author:Kevin
 * @create: 2023-10-24 17:13
 * @Description:    实现动态线程池对象
 */

public class DtpExecutor extends ThreadPoolExecutor {
    public DtpExecutor(int corePoolSize, int maximumPoolSize) {
        super(corePoolSize, maximumPoolSize, 0, TimeUnit.SECONDS, new ArrayBlockingQueue<>(10));
    }
}

创建动态线程池核心配置类

1.相关Bean的注入 2. nacos监听的bean注入

package com.laoyang.dtp;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.env.Environment;

/**
 * @author:Kevin
 * @create: 2023-10-24 18:36
 * @Description: 动态线程池核心配置类
 */
@Configuration
public class DtpExecutorAutoConfiguration {

    @Autowired
    private Environment environment;

    //最大核心数
    private static final String CORE_POOL_SIZE = "dtp.core-pool-size";
    //最大线程数
    private static final String MAXIMUM_POOL_SIZE = "dtp.maximum-pool-size";

    //创建动态线程池对象
    @Bean
    public DtpExecutor executor(){

        Integer corePoolSize = Integer.valueOf(environment.getProperty(CORE_POOL_SIZE));
        Integer maximumPoolSize = Integer.valueOf(environment.getProperty(MAXIMUM_POOL_SIZE));

        return new DtpExecutor(corePoolSize,maximumPoolSize);
    }

    @Bean
    public NacosLinsenter NacosLinsenter(){
        return new NacosLinsenter();
    }


}

然后通过springboot的自动配置实现将核心配置类注入

然后写入以下代码

org.springframework.boot.autoconfigure.EnableAutoConfiguration=com.laoyang.dtp.DtpExecutorAutoConfiguration

创建nacos的监听类并实现动态绑定

通过nacos的 Listener 接口实现相应的方法编写动态变换逻辑，同时实现spring提供的

InitializingBean接口将当前监听类通过nacos的ConfigService的addListener（）方法与dataId一一绑定。（只要dataId的配置文件发生改变，当前绑定的监听类就会调用相应的方法），最终注入线程池对象Bean，将修改的配置文件值再注入进线程池对象Bean，就实现动态线程池。

getExecutor()	创建一个线程池供下面的调用
receiveConfigInfo()	每次当前的dataId只要改变，就会调用这个方法

package com.laoyang.dtp;

import com.alibaba.nacos.api.annotation.NacosInjected;
import com.alibaba.nacos.api.config.ConfigService;
import com.alibaba.nacos.api.config.listener.Listener;
import org.springframework.beans.factory.InitializingBean;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.config.YamlPropertiesFactoryBean;
import org.springframework.core.io.ByteArrayResource;

import java.util.Properties;
import java.util.concurrent.Executor;
import java.util.concurrent.Executors;


/**
 * @author:Kevin
 * @create: 2023-10-24 19:56
 * @Description: nacos自带监听器
 */

public class NacosLinsenter implements Listener, InitializingBean {

    @NacosInjected
    private ConfigService configService;

    @Autowired
    private DtpExecutor executor;

    //nacos的dataId的名称
    private static final String DATA_ID = "dtp.yaml";

    private static final String GROUP = "DEFAULT_GROUP";
    //最大核心数
    private static final String CORE_POOL_SIZE = "dtp.core-pool-size";
    //最大线程数
    private static final String MAXIMUM_POOL_SIZE = "dtp.maximum-pool-size";


    //创建一个线程池供下面的调用
    @Override
    public Executor getExecutor() {
        return Executors.newFixedThreadPool(1);
    }

    //每次当前的dataId只要改变，就会调用这个方法
    //但是调用这个方法的线程需要上面的方法创建一个线程池
    @Override
    public void receiveConfigInfo(String s) {
        //首先需要将字符串yml格式转化为map的格式
        YamlPropertiesFactoryBean factoryBean = new YamlPropertiesFactoryBean();
        factoryBean.setResources(new ByteArrayResource(s.getBytes()));
        //使用springboot内置工具类转换为Properties类似于map的格式
        Properties object = factoryBean.getObject();
        //获取更新后的数据
        String corePoolSize = object.getProperty(CORE_POOL_SIZE);
        String maximumPoolSize = object.getProperty(MAXIMUM_POOL_SIZE);
        //直接更新数据
        executor.setCorePoolSize(Integer.parseInt(corePoolSize));
        executor.setMaximumPoolSize(Integer.parseInt(maximumPoolSize));

    }

    @Override
    public void afterPropertiesSet() throws Exception {
        //将这个NacosLinsenter与当前的dataId一一绑定
        configService.addListener(DATA_ID,GROUP,this);
    }
}

3. 开始在user模块使用 dtp-spring-boot-starter模块

创建启动Springboot配置类

package com.laoyang;

import com.alibaba.nacos.spring.context.annotation.config.NacosPropertySource;
import com.alibaba.nacos.spring.context.annotation.discovery.EnableNacosDiscovery;
import com.laoyang.dtp.DtpExecutor;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.core.env.Environment;


/**
 * @author:Kevin
 * @create: 2023-10-24 17:04
 * @Description:
 */
@SpringBootApplication
@NacosPropertySource(dataId = "dtp.yaml", autoRefreshed = true)
public class UserApplication {

    public static void main(String[] args) {
        SpringApplication.run(UserApplication.class, args);
    }
}

创建Controller注入动态线程池对象来使用

package com.laoyang.Controller;

import com.laoyang.dtp.DtpExecutor;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.GetMapping;

import java.util.concurrent.ThreadPoolExecutor;

/**
 * @author:Kevin
 * @create: 2023-10-24 17:05
 * @Description: 视图层
 */
@Controller
public class UserController {

    @Autowired
    private DtpExecutor executor;

    @GetMapping
    public Integer test(){
        executor.execute(() -> dotest());
        return 1;
    }

    public void dotest(){
        System.out.println("dotest");
    }


}

到此大功告成！！

二，改进

优化1：假如我们没有配置核心线程数或者最大线程数的话会报错，所以我们要优雅的创建默认值。

步骤：创建配置文件对象（

@ConfigurationProperties("dtp")这个注解会根据参数，找到nacos的配置文件的yml格式的字段，并变成Bean对象。

）

package com.laoyang.dtp;

import org.springframework.boot.context.properties.ConfigurationProperties;

/**
 * @author:Kevin
 * @create: 2023-10-24 21:29
 * @Description: 创建配置文件对象
 */
@ConfigurationProperties("dtp")
public class DtpProperties {

    private String corePoolSize = "10";
    private String maximumPoolSize = "100";

    public String getCorePoolSize() {
        return corePoolSize;
    }

    public void setCorePoolSize(String corePoolSize) {
        this.corePoolSize = corePoolSize;
    }

    public String getMaximumPoolSize() {
        return maximumPoolSize;
    }

    public void setMaximumPoolSize(String maximumPoolSize) {
        this.maximumPoolSize = maximumPoolSize;
    }
}

然后在DtpExecutorAutoConfiguration核心配置类中加上@EnableConfigurationProperties(DtpProperties.class) 注解

然后通过传参的形式优化

优化2. 上面的只能实现一个线程池对象，但是实际项目中并不只是这一个线程池对象，所以接下来我们需要进行优化！

创建一个DtpUtil 将来用来存放创建的多个线程池对象

package com.laoyang.dtp;

import java.util.HashMap;
import java.util.concurrent.ConcurrentHashMap;

/**
 * @author:Kevin
 * @create: 2023-10-24 22:29
 * @Description:    线程安全的ConcurrentHashMap
 */

public class DtpUtil {

//    public static ConcurrentHashMap<String,DtpExecutor> map = new ConcurrentHashMap<>();
    public static HashMap<String,DtpExecutor> map = new HashMap<>();

    public static void set(String name,DtpExecutor dtpExecutor){
        map.put(name,dtpExecutor);
    }

    public static DtpExecutor get(String name) {
        return map.get(name);
    }

}

实现手动注册bean对象

package com.laoyang.dtp;

import org.springframework.beans.factory.support.AbstractBeanDefinition;
import org.springframework.beans.factory.support.BeanDefinitionBuilder;
import org.springframework.beans.factory.support.BeanDefinitionRegistry;
import org.springframework.beans.factory.support.BeanNameGenerator;
import org.springframework.boot.context.properties.bind.Bindable;
import org.springframework.boot.context.properties.bind.Binder;
import org.springframework.context.EnvironmentAware;
import org.springframework.context.annotation.ImportBeanDefinitionRegistrar;
import org.springframework.core.ResolvableType;
import org.springframework.core.env.Environment;
import org.springframework.core.type.AnnotationMetadata;

/**
 * @author:Kevin
 * @create: 2023-10-24 21:57
 * @Description: 注册配置文件bean
 */

public class DtpImportBeanDefinationRegister implements ImportBeanDefinitionRegistrar, EnvironmentAware {

    //传入配置文件对象
    private Environment environment;

    @Override
    public void registerBeanDefinitions(AnnotationMetadata importingClassMetadata, BeanDefinitionRegistry registry, BeanNameGenerator importBeanNameGenerator) {
        //注册bean,读取用户配置的dtp后面定义的线程池列表，然后转换为DtpProperties对象
        DtpProperties dtpProperties = new DtpProperties();

        Binder binder = Binder.get(environment);
        ResolvableType type = ResolvableType.forClass(DtpProperties.class);
        Bindable<?> target = Bindable.of(type).withExistingValue(dtpProperties);
        binder.bind("dtp",target);

        //遍历配置，拿到所有的线程池列表
        for (DtpProperties.DtpExecutorProperties executorProperties : dtpProperties.getDtpExecutors()) {
            AbstractBeanDefinition beanDefinition = BeanDefinitionBuilder.genericBeanDefinition().getBeanDefinition();
            beanDefinition.setBeanClass(DtpExecutor.class);
            //往这个DtpExecutor的有参函数中传入两个值
            beanDefinition.getConstructorArgumentValues().addGenericArgumentValue(executorProperties.getCorePoolSize());
            beanDefinition.getConstructorArgumentValues().addGenericArgumentValue(executorProperties.getMaximumPoolSize());

            registry.registerBeanDefinition(executorProperties.getName(),beanDefinition);

        }


    }

    //
    @Override
    public void setEnvironment(Environment environment) {
        this.environment = environment;
    }
}

同时不要忘了在核心配置类注入这个配置

创建bean的后置处理器

package com.laoyang.dtp;

import org.springframework.beans.BeansException;
import org.springframework.beans.factory.config.BeanPostProcessor;

/**
 * @author:Kevin
 * @create: 2023-10-25 10:29
 * @Description: bean的后置处理器：用于注入我们的多动态线程池对象
 */

public class DtpBeanPostProcessor implements BeanPostProcessor {

    /**
     *         注入这个对象 DtpExecutor
     * @param bean
     * @param beanName
     * @return
     * @throws BeansException
     */
    @Override
    public Object postProcessAfterInitialization(Object bean, String beanName) throws BeansException {

        if (bean instanceof DtpExecutor){
            DtpUtil.set(beanName,(DtpExecutor)bean);
        }

        return bean;
    }
}

也需要导入该配置类

最后在监听类修改代码（NacosLinsenter）

package com.laoyang.dtp;

import com.alibaba.nacos.api.annotation.NacosInjected;
import com.alibaba.nacos.api.config.ConfigService;
import com.alibaba.nacos.api.config.listener.Listener;
import org.springframework.beans.factory.InitializingBean;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.config.YamlPropertiesFactoryBean;
import org.springframework.boot.context.properties.bind.Bindable;
import org.springframework.boot.context.properties.bind.Binder;
import org.springframework.boot.context.properties.source.MapConfigurationPropertySource;
import org.springframework.core.ResolvableType;
import org.springframework.core.io.ByteArrayResource;

import java.util.List;
import java.util.Properties;
import java.util.concurrent.Executor;
import java.util.concurrent.Executors;


/**
 * @author:Kevin
 * @create: 2023-10-24 19:56
 * @Description: nacos自带监听器
 */

public class NacosLinsenter implements Listener, InitializingBean {

    @NacosInjected
    private ConfigService configService;

    //nacos的dataId的名称
    private static final String DATA_ID = "dtp.yaml";

    private static final String GROUP = "DEFAULT_GROUP";


    //创建一个线程池供下面的调用
    @Override
    public Executor getExecutor() {
        return Executors.newFixedThreadPool(1);
    }

    //每次当前的dataId只要改变，就会调用这个方法
    //但是调用这个方法的线程需要上面的方法创建一个线程池
    @Override
    public void receiveConfigInfo(String s) {
        //首先需要将字符串yml格式转化为map的格式(单个线程池)
        YamlPropertiesFactoryBean factoryBean = new YamlPropertiesFactoryBean();
        factoryBean.setResources(new ByteArrayResource(s.getBytes()));
        //使用springboot内置工具类转换为Properties类似于map的格式
        Properties properties = factoryBean.getObject();

        //首先需要将字符串yml格式转化为List的格式(多个线程池)
        DtpProperties dtpProperties = new DtpProperties();
        MapConfigurationPropertySource sources = new MapConfigurationPropertySource(properties);
        Binder binder = new Binder(sources);
        ResolvableType type = ResolvableType.forClass(DtpProperties.class);
        Bindable<?> target = Bindable.of(type).withExistingValue(dtpProperties);
        binder.bind("dtp",target);

        //获取到更改配置的内容
        List<DtpProperties.DtpExecutorProperties> executors = dtpProperties.getDtpExecutors();

        //将改过的数据注入bean中
        for (DtpProperties.DtpExecutorProperties executor : executors) {
            //获取bean对象
            DtpExecutor dtpExecutor = DtpUtil.get(executor.getName());
            //然后修改数据
            dtpExecutor.setCorePoolSize(executor.getCorePoolSize());
            dtpExecutor.setMaximumPoolSize(executor.getMaximumPoolSize());

        }

        //获取更新后的数据
//        String corePoolSize = properties.getProperty(CORE_POOL_SIZE);
//        String maximumPoolSize = properties.getProperty(MAXIMUM_POOL_SIZE);
//        //直接更新数据
//        executor.setCorePoolSize(Integer.parseInt(corePoolSize));
//        executor.setMaximumPoolSize(Integer.parseInt(maximumPoolSize));

    }

    @Override
    public void afterPropertiesSet() throws Exception {
        //将这个NacosLinsenter与当前的dataId一一绑定
        configService.addListener(DATA_ID,GROUP,this);
    }
}

最后在nacos修改为多个配置

优化3：实现告警功能

创建DtpMonitor监控类

package com.laoyang.dtp;

import org.springframework.beans.factory.InitializingBean;

import java.util.Map;
import java.util.concurrent.ScheduledThreadPoolExecutor;
import java.util.concurrent.TimeUnit;

/**
 * @author:Kevin
 * @create: 2023-10-25 11:05
 * @Description: 优化之监听器：
 *                  监听活跃的线程数
 *                  开始定时任务
 */

public class DtpMonitor implements InitializingBean {

    ScheduledThreadPoolExecutor executor = new ScheduledThreadPoolExecutor(1);

    @Override
    public void afterPropertiesSet() throws Exception {
        //每隔5秒执行一次这个方法
        executor.scheduleAtFixedRate(() -> {

            for (Map.Entry<String, DtpExecutor> entry : DtpUtil.map.entrySet()) {
                String name = entry.getKey();
                DtpExecutor exectuor = entry.getValue();

                //线程池的活跃数量
                int activeCount = exectuor.getActiveCount();

                if (activeCount > 20){
                    //TODO 线程池活跃线程池告警
                    System.out.println("告警！");
                }
            }

        },5,5, TimeUnit.SECONDS);

    }
}

同时在核心配置类注入当前配置

至此，项目集成完毕！！！

玛卡巴咖

关注

0
点赞
踩
1

收藏

觉得还不错? 一键收藏
0
评论
美团动态ThreadPoolExecutor底层实现源码实战

介绍springboot连接nacos实现动态线程池，同时得安装nacos，同时代码将有两个模块，dtp-spring-boot-starter 与 user 模块，前者将是独立的动态线程池，可以引入自己的项目中，后者模块主要用于测试与使用动态线程池模块。注意springboot与nacos的适配版本！
复制链接

扫一扫