springboot自带的线程池ThreadPoolTaskExecutor、ThreadPoolTaskScheduler的深入应用——异步任务监听回调,任务中断案例

一、常用的的线程池对象

1.jdk原生的两个常用线程池对象
ThreadPoolExecutor、ScheduledThreadPoolExecutor，后者继承前者，主要增加了任务调度相关的一些方法
2.springboot自动装配的两个常用线程池对象
如果引入了spring-boot-autoconfigure这个依赖，则会自动装配两个线程池对象ThreadPoolTaskExecutor，ThreadPoolTaskScheduler（参考org.springframework.boot.autoconfigure.task.TaskExecutionAutoConfiguration。），分别对应jdk的两个线程池，是静态代理增强，故ThreadPoolTaskScheduler的api是最丰富的。

二、ThreadPoolTaskScheduler核心api测试

    @Autowired
    private ThreadPoolTaskScheduler taskScheduler;
    @Autowired
    private ThreadPoolTaskExecutor taskExecutor;

1.监听异步任务，如下dateTime即是任务异步执行成功的回调函数入参，后一个失败回调函数。

 public void test1() {
        taskScheduler.submitListenable(() -> {
            //int a = 1 / 0;
            TimeUnit.SECONDS.sleep(3);
            return LocalDateTime.now();
        }).addCallback(dateTime -> System.out.println(dateTime),
                e -> System.out.println(e)
        );
        System.out.println("end");
    }

输出：

end
2020-11-21T10:42:41.511

2. 普通任务提交。默认1个线程，无界队列，可通过spring.tasks.cheduling.pool.size配置并发数

public void test2(int size) {
        for (int i = 0; i < size; i++) {
            int finalI = i;
            taskScheduler.submit(() -> {
                try {
                    TimeUnit.SECONDS.sleep(3);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
                ScheduledThreadPoolExecutor poolExecutor = taskScheduler.getScheduledThreadPoolExecutor();
                System.out.println("ActiveCount"+ poolExecutor.getActiveCount());
                System.out.println("CompletedTaskCount"+ poolExecutor.getCompletedTaskCount());
                System.out.println("LargestPoolSize"+ poolExecutor.getLargestPoolSize());
                System.out.println("PoolSize"+ poolExecutor.getPoolSize());
                System.out.println("QueueSize"+ poolExecutor.getQueue().size());
                System.out.println("TaskCount"+ poolExecutor.getTaskCount());
                System.out.println("RejectedExecutionHandler"+ poolExecutor.getRejectedExecutionHandler().toString());
                System.out.println(finalI);
           });

        }
    }

3.定时任务提交。包括一次性任务，周期性任务及延迟任务。

区别：scheduleAtFixedRate与scheduleWithFixedDelay

前者指定时延后开始执行任务，以后每隔period的时长再次执行该任务 ，如果前一个任务到了period仍未结速则等待其结束后立即执行，如前一任务提前完成则需要等待到period才执行下一任务。

后者指定时延后开始执行任务，上一个任务【完成后】必须等待delay时长，再次执行任务。

scheduleAtFixedRate与scheduleAtFixedRate区别主要是前者等待间隔不是确定的，二后者是固定的。

延迟10s执行，可从ScheduledFutrue获取剩余延迟。

 public void test3() throws InterruptedException {
        ScheduledFuture<?> schedule = taskScheduler.schedule(() -> {
            System.out.println(LocalDateTime.now());
        }, Instant.now().plusSeconds(10));

        System.out.println(schedule.getDelay(TimeUnit.SECONDS));
    }

4.invokeAll和invokeAny

 public void test3() throws InterruptedException, ExecutionException {
        List<Future<Object>> futures = scheduledPoolExecutor.invokeAll(Lists.newArrayList(() -> {
                    TimeUnit.SECONDS.sleep(1);
                    return 1;
                }, () -> {
                    TimeUnit.SECONDS.sleep(2);
                    return 2;
                }, () -> {
                    TimeUnit.SECONDS.sleep(6);
                    return 3;
                }
        ), 10, TimeUnit.SECONDS);
        for (Future<Object> future : futures) {
            System.out.println(future.get());
        }
    }

三、ThreadPoolTaskExecutor核心api测试

较ThreadPoolTaskScheduler少了schedule相关api,但是配置项可配置线程池几个核心参数，但是拒绝策略不可配置,使用默认的AbortPolicy并抛出异常

测试配置

---
spring:
  task:
    execution:
      pool:
        core-size: 1
        max-size: 2
        queue-capacity: 1

测试代码

public void test5( int size) throws InterruptedException {
        for (int i = 0; i < size; i++) {
            int finalI = i;
            taskExecutor.submit(() -> {
                try {
                    TimeUnit.SECONDS.sleep(3);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
                ThreadPoolExecutor poolExecutor = taskExecutor.getThreadPoolExecutor();
                System.out.println("PoolSize" + poolExecutor.getPoolSize());
                System.out.println("ActiveCount" + poolExecutor.getActiveCount());
                System.out.println("LargestPoolSize" + poolExecutor.getLargestPoolSize());
                System.out.println("MaximumPoolSize" + poolExecutor.getMaximumPoolSize());
                System.out.println("KeepAliveTime" + poolExecutor.getKeepAliveTime(TimeUnit.SECONDS));
                System.out.println("ThreadFactory" + poolExecutor.getThreadFactory().toString());
                System.out.println("QueueSize" + poolExecutor.getQueue().size());
                System.out.println("TaskCount" + poolExecutor.getTaskCount());
                System.out.println("CompletedTaskCount" + poolExecutor.getCompletedTaskCount());
                System.out.println("RejectedExecutionHandler" + poolExecutor.getRejectedExecutionHandler().toString());
                System.out.println("任务index:" + finalI);
            });
        }
    }

输出如下：提交任务size=4. 任务1直接执行，任务2进入queue，任务3发现queue已满，创建新线程直接执行，任务3发现queue已满并且线程池已是maxsize,故抛出了拒绝异常。输出任务index:0=>任务index:2=>任务index:1符合预期

Caused by: java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5a35ac51 rejected from java.util.concurrent.ThreadPoolExecutor@f179466[Running, pool size = 2, active threads = 1, queued tasks = 1, completed tasks = 0]
    at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2047)
    at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823)
    at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1369)
    at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112)
    at org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor.submit(ThreadPoolTaskExecutor.java:330)
    ... 71 common frames omitted
[2020-11-21 12:12:26][container-2][ERROR][org.springframework.data.redis.listener.RedisMessageListenerContainer:651]
 Connection failure occurred. Restarting subscription task after 5000 ms
PoolSize2
ActiveCount2
LargestPoolSize2
MaximumPoolSize2
KeepAliveTime60
ThreadFactoryorg.springframework.scheduling.concurrent.ThreadPoolTaskExecutor@5ee6fdc4
QueueSize1
TaskCount3
CompletedTaskCount0
RejectedExecutionHandlerjava.util.concurrent.ThreadPoolExecutor$AbortPolicy@32cb169
任务index:0
PoolSize2
ActiveCount2
LargestPoolSize2
MaximumPoolSize2
KeepAliveTime60
ThreadFactoryorg.springframework.scheduling.concurrent.ThreadPoolTaskExecutor@5ee6fdc4
QueueSize0
TaskCount3
CompletedTaskCount1
RejectedExecutionHandlerjava.util.concurrent.ThreadPoolExecutor$AbortPolicy@32cb169
任务index:2
PoolSize2
ActiveCount1
LargestPoolSize2
MaximumPoolSize2
KeepAliveTime60
ThreadFactoryorg.springframework.scheduling.concurrent.ThreadPoolTaskExecutor@5ee6fdc4
QueueSize0
TaskCount3
CompletedTaskCount2
RejectedExecutionHandlerjava.util.concurrent.ThreadPoolExecutor$AbortPolicy@32cb169
任务index:1

四、使用案例

需求：任务业务耗时稍长，不宜循环串行处理。异步处理时要求（park主线程），任何一个任务异常或全部正常完成即可进行后续业务(unpark主线程)。有一个异步任务异常后要终止所有的异步任务(shodownNow)，并且支持多次执行(关闭后再次initialize)

注意点：

1.主线程阻塞这里使用park/unpark，countdownlatch也是适合的；

2.业务代码异常如果被捕获是进入正常回调，否则进入异常回调，异常回调如果再将异常抛出是不会抛到主线程的，故需要定义一个计数器；

3.不宜使用FutrueTask的get阻塞方法获取异步任务结果或循环检查异步任务是否完成再获取结果，

4.代码结构与实际效果很类似js的promise: xxx=>then(xx=>{}).catch(e=>{xxx})

5.要中断进行线程池进行中的线程需要shutdownNow而不是shoutdown,关闭后记得initialize否则这个线程池不能重复利用

package com.xxl.job.admin;


import groovy.util.logging.Slf4j;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;
import org.springframework.util.Assert;

import java.util.*;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.locks.LockSupport;

/**
 * Description: TODO
 *
 * @author majun
 * @version 1.0
 * @date 2019-09-16 18:38
 */
public class Test {
       static Logger log=    LoggerFactory.getLogger("xxx");

    //springboot项目考虑@Autowire,测试方便自己创建一个
    private static ThreadPoolTaskExecutor executor;

    static {
        executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(10);
        executor.setMaxPoolSize(10);
        executor.initialize();
    }

    public static void main(String[] args) {

        AtomicInteger finishCounter = new AtomicInteger(0);
        List<Integer> taskIds = Arrays.asList(new Integer[]{7,8,9});
        Thread t = Thread.currentThread();
        taskIds.forEach(id -> {
            executor.submitListenable(() -> {
                int random = 8;
                if (random < id) {
                    throw new RuntimeException("业务异常ID=" + id);
                } else {
                    Thread.sleep(10000);
                    log.info("业务正常的ID=" + id);
                    return "业务正常的ID=" + id;
                }

            }).addCallback(s -> {
                int currentSuccess = finishCounter.incrementAndGet();
                if (currentSuccess == taskIds.size()) {
                    log.info("全部异步任务都正常完成了");
                    LockSupport.unpark(t);
                }
            }, e -> {
                log.error("有异步异常了", e);
                LockSupport.unpark(t);
            });
        });
        log.info("主函数已阻塞");
        LockSupport.park(t);
        log.info("主函数已解除阻塞");
        log.info("异步任务完成成功数" + finishCounter.get());
        log.info("线程池当前使用的线程{},已完成任务数{}",executor.getActiveCount(),executor.getThreadPoolExecutor().getCompletedTaskCount());
        executor.getThreadPoolExecutor().shutdownNow();//线程池立即关闭，停止所有任务包括进行中的任务，这里直接关闭，进行中异步任务sleep中会抛出中断异常
        executor.initialize();//重新初始化线程池后可接受新的异步任务
        log.info("线程池当前使用的线程{},已完成任务数{}",executor.getActiveCount(),executor.getThreadPoolExecutor().getCompletedTaskCount());
        executor.execute(() -> System.out.println("重启线程池后的新异步任务执行了"));
        Assert.isTrue(finishCounter.get()==taskIds.size(),"异步任务有异常的,不必进行后续业务了");
        log.info("开始后续业务");
    }


}

D:\jdk8\bin\java.exe -agentlib:jdwp=transport=dt_socket,address=127.0.0.1:53639,suspend=y,server=n -javaagent:C:\Users\Administrator\.IntelliJIdea2018.2\system\captureAgent\debugger-agent.jar=file:/C:/Users/Administrator/AppData/Local/Temp/capture.props -Dfile.encoding=UTF-8 -classpath "D:\jdk8\jre\lib\charsets.jar;D:\jdk8\jre\lib\deploy.jar;D:\jdk8\jre\lib\ext\access-bridge-64.jar;D:\jdk8\jre\lib\ext\cldrdata.jar;D:\jdk8\jre\lib\ext\dnsns.jar;D:\jdk8\jre\lib\ext\jaccess.jar;D:\jdk8\jre\lib\ext\jfxrt.jar;D:\jdk8\jre\lib\ext\localedata.jar;D:\jdk8\jre\lib\ext\nashorn.jar;D:\jdk8\jre\lib\ext\sunec.jar;D:\jdk8\jre\lib\ext\sunjce_provider.jar;D:\jdk8\jre\lib\ext\sunmscapi.jar;D:\jdk8\jre\lib\ext\sunpkcs11.jar;D:\jdk8\jre\lib\ext\zipfs.jar;D:\jdk8\jre\lib\javaws.jar;D:\jdk8\jre\lib\jce.jar;D:\jdk8\jre\lib\jfr.jar;D:\jdk8\jre\lib\jfxswt.jar;D:\jdk8\jre\lib\jsse.jar;D:\jdk8\jre\lib\management-agent.jar;D:\jdk8\jre\lib\plugin.jar;D:\jdk8\jre\lib\resources.jar;D:\jdk8\jre\lib\rt.jar;D:\xxl-job2\xxl-job-admin\target\classes;D:\mymavenrepository\org\springframework\boot\spring-boot-starter-web\2.4.0\spring-boot-starter-web-2.4.0.jar;D:\mymavenrepository\org\springframework\boot\spring-boot-starter\2.4.0\spring-boot-starter-2.4.0.jar;D:\mymavenrepository\org\springframework\boot\spring-boot\2.4.0\spring-boot-2.4.0.jar;D:\mymavenrepository\org\springframework\boot\spring-boot-autoconfigure\2.4.0\spring-boot-autoconfigure-2.4.0.jar;D:\mymavenrepository\org\springframework\boot\spring-boot-starter-logging\2.4.0\spring-boot-starter-logging-2.4.0.jar;D:\mymavenrepository\ch\qos\logback\logback-classic\1.2.3\logback-classic-1.2.3.jar;D:\mymavenrepository\ch\qos\logback\logback-core\1.2.3\logback-core-1.2.3.jar;D:\mymavenrepository\org\apache\logging\log4j\log4j-to-slf4j\2.13.3\log4j-to-slf4j-2.13.3.jar;D:\mymavenrepository\org\apache\logging\log4j\log4j-api\2.13.3\log4j-api-2.13.3.jar;D:\mymavenrepository\org\slf4j\jul-to-slf4j\1.7.30\jul-to-slf4j-1.7.30.jar;D:\mymavenrepository\jakarta\annotation\jakarta.annotation-api\1.3.5\jakarta.annotation-api-1.3.5.jar;D:\mymavenrepository\org\yaml\snakeyaml\1.27\snakeyaml-1.27.jar;D:\mymavenrepository\org\springframework\boot\spring-boot-starter-json\2.4.0\spring-boot-starter-json-2.4.0.jar;D:\mymavenrepository\com\fasterxml\jackson\core\jackson-databind\2.11.3\jackson-databind-2.11.3.jar;D:\mymavenrepository\com\fasterxml\jackson\core\jackson-annotations\2.11.3\jackson-annotations-2.11.3.jar;D:\mymavenrepository\com\fasterxml\jackson\core\jackson-core\2.11.3\jackson-core-2.11.3.jar;D:\mymavenrepository\com\fasterxml\jackson\datatype\jackson-datatype-jdk8\2.11.3\jackson-datatype-jdk8-2.11.3.jar;D:\mymavenrepository\com\fasterxml\jackson\datatype\jackson-datatype-jsr310\2.11.3\jackson-datatype-jsr310-2.11.3.jar;D:\mymavenrepository\com\fasterxml\jackson\module\jackson-module-parameter-names\2.11.3\jackson-module-parameter-names-2.11.3.jar;D:\mymavenrepository\org\springframework\boot\spring-boot-starter-tomcat\2.4.0\spring-boot-starter-tomcat-2.4.0.jar;D:\mymavenrepository\org\apache\tomcat\embed\tomcat-embed-core\9.0.39\tomcat-embed-core-9.0.39.jar;D:\mymavenrepository\org\glassfish\jakarta.el\3.0.3\jakarta.el-3.0.3.jar;D:\mymavenrepository\org\apache\tomcat\embed\tomcat-embed-websocket\9.0.39\tomcat-embed-websocket-9.0.39.jar;D:\mymavenrepository\org\springframework\spring-web\5.3.1\spring-web-5.3.1.jar;D:\mymavenrepository\org\springframework\spring-beans\5.3.1\spring-beans-5.3.1.jar;D:\mymavenrepository\org\springframework\spring-webmvc\5.3.1\spring-webmvc-5.3.1.jar;D:\mymavenrepository\org\springframework\spring-aop\5.3.1\spring-aop-5.3.1.jar;D:\mymavenrepository\org\springframework\spring-context\5.3.1\spring-context-5.3.1.jar;D:\mymavenrepository\org\springframework\spring-expression\5.3.1\spring-expression-5.3.1.jar;D:\mymavenrepository\org\springframework\spring-core\5.3.1\spring-core-5.3.1.jar;D:\mymavenrepository\org\springframework\spring-jcl\5.3.1\spring-jcl-5.3.1.jar;D:\mymavenrepository\org\springframework\boot\spring-boot-starter-freemarker\2.4.0\spring-boot-starter-freemarker-2.4.0.jar;D:\mymavenrepository\org\freemarker\freemarker\2.3.30\freemarker-2.3.30.jar;D:\mymavenrepository\org\springframework\spring-context-support\5.3.1\spring-context-support-5.3.1.jar;D:\mymavenrepository\org\springframework\boot\spring-boot-starter-mail\2.4.0\spring-boot-starter-mail-2.4.0.jar;D:\mymavenrepository\com\sun\mail\jakarta.mail\1.6.5\jakarta.mail-1.6.5.jar;D:\mymavenrepository\com\sun\activation\jakarta.activation\1.2.2\jakarta.activation-1.2.2.jar;D:\mymavenrepository\org\springframework\boot\spring-boot-starter-actuator\2.4.0\spring-boot-starter-actuator-2.4.0.jar;D:\mymavenrepository\org\springframework\boot\spring-boot-actuator-autoconfigure\2.4.0\spring-boot-actuator-autoconfigure-2.4.0.jar;D:\mymavenrepository\org\springframework\boot\spring-boot-actuator\2.4.0\spring-boot-actuator-2.4.0.jar;D:\mymavenrepository\io\micrometer\micrometer-core\1.6.1\micrometer-core-1.6.1.jar;D:\mymavenrepository\org\hdrhistogram\HdrHistogram\2.1.12\HdrHistogram-2.1.12.jar;D:\mymavenrepository\org\latencyutils\LatencyUtils\2.0.3\LatencyUtils-2.0.3.jar;D:\mymavenrepository\org\mybatis\spring\boot\mybatis-spring-boot-starter\2.1.4\mybatis-spring-boot-starter-2.1.4.jar;D:\mymavenrepository\org\springframework\boot\spring-boot-starter-jdbc\2.4.0\spring-boot-starter-jdbc-2.4.0.jar;D:\mymavenrepository\com\zaxxer\HikariCP\3.4.5\HikariCP-3.4.5.jar;D:\mymavenrepository\org\springframework\spring-jdbc\5.3.1\spring-jdbc-5.3.1.jar;D:\mymavenrepository\org\springframework\spring-tx\5.3.1\spring-tx-5.3.1.jar;D:\mymavenrepository\org\mybatis\spring\boot\mybatis-spring-boot-autoconfigure\2.1.4\mybatis-spring-boot-autoconfigure-2.1.4.jar;D:\mymavenrepository\org\mybatis\mybatis\3.5.6\mybatis-3.5.6.jar;D:\mymavenrepository\org\mybatis\mybatis-spring\2.0.6\mybatis-spring-2.0.6.jar;D:\mymavenrepository\mysql\mysql-connector-java\8.0.22\mysql-connector-java-8.0.22.jar;D:\xxl-job2\xxl-job-core\target\classes;D:\mymavenrepository\io\netty\netty-all\4.1.54.Final\netty-all-4.1.54.Final.jar;D:\mymavenrepository\com\google\code\gson\gson\2.8.6\gson-2.8.6.jar;D:\mymavenrepository\org\codehaus\groovy\groovy\2.5.13\groovy-2.5.13.jar;D:\mymavenrepository\org\slf4j\slf4j-api\1.7.30\slf4j-api-1.7.30.jar;C:\Program Files\JetBrains\IntelliJ IDEA 2018.2.5\lib\idea_rt.jar" com.xxl.job.admin.Test
Connected to the target VM, address: '127.0.0.1:53639', transport: 'socket'
22:11:28.297 logback [main] INFO  o.s.s.c.ThreadPoolTaskExecutor - Initializing ExecutorService
22:11:28.434 logback [main] INFO  xxx - 主函数已阻塞
22:11:28.442 logback [ThreadPoolTaskExecutor-3] ERROR xxx - 有异步任务异常了
java.lang.RuntimeException: 业务异常ID=9
    at com.xxl.job.admin.Test.lambda$null$0(Test.java:44)
    at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
    at java.util.concurrent.FutureTask.run(FutureTask.java)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:748)
22:11:28.443 logback [main] INFO  xxx - 主函数已解除阻塞
22:11:28.444 logback [main] INFO  xxx - 异步任务完成成功数0
22:11:28.444 logback [main] INFO  xxx - 线程池当前使用的线程2,已完成任务数1
22:11:28.449 logback [main] INFO  o.s.s.c.ThreadPoolTaskExecutor - Initializing ExecutorService
22:11:28.449 logback [main] INFO  xxx - 线程池当前使用的线程0,已完成任务数0
22:11:28.449 logback [ThreadPoolTaskExecutor-1] ERROR xxx - 有异步任务异常了
java.lang.InterruptedException: sleep interrupted
    at java.lang.Thread.sleep(Native Method)
    at com.xxl.job.admin.Test.lambda$null$0(Test.java:46)
    at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
    at java.util.concurrent.FutureTask.run(FutureTask.java)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:748)
22:11:28.450 logback [ThreadPoolTaskExecutor-2] ERROR xxx - 有异步任务异常了
java.lang.InterruptedException: sleep interrupted
    at java.lang.Thread.sleep(Native Method)
    at com.xxl.job.admin.Test.lambda$null$0(Test.java:46)
    at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
    at java.util.concurrent.FutureTask.run(FutureTask.java)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:748)
重启线程池后的新异步任务执行了
Exception in thread "main" java.lang.IllegalArgumentException: 异步任务有异常的,不必进行后续业务了
    at org.springframework.util.Assert.isTrue(Assert.java:121)
    at com.xxl.job.admin.Test.main(Test.java:71)

五、自定义线程池

如果需要优先级队列PriorityBlockingQueue，自定义拒绝策略等则需要自定义线程池，但是会失去springboot的submitListenable异步任务回调等相关的api,当然这可参照spring的源码自己去实现。@SpringBootApplication(exclude = {TaskExecutionAutoConfiguration.class})排除 spring-boot-autoconfigure自动装配的线程池，然后配置一个线程池对象到spring容器。

其它文档：https://blog.csdn.net/luanmousheng/article/details/77816412