并发编程应用场景的时候经常遇到处理批量任务的情况,那么使用ThreadPoolExecutor#invokeAll、或者使用CountDownLatch协调,可以见上一篇的项目使用场景,当然CompletionService本身也可以实现。更复杂的场景:当执行批量任务回来,每个都还有后续的操作,并行不想等到所以结果返回再处理后续的结果,而是先回来就先处理,那么要是我们自己会怎么处理呢,可以将其放入线程安全的队列,阻塞获取队列的结果,比如:
/**
* 批量执行任务
*
* 保存价格数据库了:40
* 保存价格数据库了:20
* 保存价格数据库了:30
*
* @author kevin
* @date 2020/8/4 22:54
* @since 1.0.0
*/
public class BatchTaskDemo {
public static void main(String[] args) throws ExecutionException, InterruptedException {
new BatchTaskDemo().demo();
}
/**
* ThreadPoolExecutor+Future 方式实现, 批量查询电商系统的价格,并且保存到数据库中
* 并且能够先执行完先保存,节约执行时间
*/
public void demo() throws ExecutionException, InterruptedException {
final ThreadPoolExecutor threadPoolExecutor = new ThreadPoolExecutor(100, 100,
0, TimeUnit.MILLISECONDS, new LinkedBlockingDeque<>(), new OldSimpleThreadPool.DefaultThreadFactory("test"));
final LinkedBlockingQueue<Integer> queue = new LinkedBlockingQueue<>();
threadPoolExecutor.execute(() -> {
try {
queue.put(threadPoolExecutor.submit(() -> {
sleep(300);
return 20;
}).get());
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
});
threadPoolExecutor.execute(() -> {
try {
queue.put(threadPoolExecutor.submit(() -> {
sleep(500);
return 30;
}).get());
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
});
threadPoolExecutor.execute(() ->{
try {
queue.put(threadPoolExecutor.submit(() -> {
sleep(200);
return 40;
}).get());
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
});
for (int i = 0; i < 3; i++) {
Integer take = queue.take();
System.out.println("保存价格数据库了:" + take);
}
}
public static void sleep(long timeout) {
try {
Thread.sleep(timeout);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
只是像这样的场景,官方提供了CompletionService的子类ExecutorCompletionService去解决这样的问题,先父类定义的接口:
public interface CompletionService<V> {
// 提交任务
Future<V> submit(Callable<V> var1);
Future<V> submit(Runnable var1, V var2);
// 队列为空则调用方法的线程会被阻塞
Future<V> take() throws InterruptedException;
// 可能回返回null值
Future<V> poll();
// 支持超时,如果超时了队列还是为空则返回null值
Future<V> poll(long var1, TimeUnit var3) throws InterruptedException;
}
在看看ExecutorCompletionService的构造,基本结构和原理同上面我们自己写的Demo,提供了两个构造函数。线程池一点要我们传入的(当前只是一个工具而已),队列如果我们不传入则使用默认的LinkedBlockingQueue(队列长度为Integer.MAX_VALUE),所以自己使用时一般都是自己传入。至于task和poll方法都是直接调用LinkedBlockingQueue的方法,具体在LinkedBlockingQueue篇中进行分析:
public class ExecutorCompletionService<V> implements CompletionService<V> {
private final Executor executor;
private final AbstractExecutorService aes;
private final BlockingQueue<Future<V>> completionQueue;
public ExecutorCompletionService(Executor executor) {
if (executor == null)
throw new NullPointerException();
this.executor = executor;
this.aes = (executor instanceof AbstractExecutorService) ?
(AbstractExecutorService) executor : null;
// 我们不传入使用LinkedBlockingQueue长度为Interger.MAX_VALUE
this.completionQueue = new LinkedBlockingQueue<Future<V>>();
}
public ExecutorCompletionService(Executor executor,
BlockingQueue<Future<V>> completionQueue) {
if (executor == null || completionQueue == null)
throw new NullPointerException();
this.executor = executor;
this.aes = (executor instanceof AbstractExecutorService) ?
(AbstractExecutorService) executor : null;
this.completionQueue = completionQueue;
}
}
那么实现上面的场景,代码如下(比较简约):
public class CompletionServiceDemo {
/**
* 获取线程池
*/
private static final ThreadPoolExecutor threadPoolExecutor = new ThreadPoolExecutor(100, 100,
0, TimeUnit.MILLISECONDS, new LinkedBlockingDeque<>(), new OldSimpleThreadPool.DefaultThreadFactory("test"));
/**
* 使用默认的无界队列 {@link java.util.concurrent.LinkedBlockingQueue}
*/
public static final CompletionService<Integer> COMPLETION_SERVICE = new ExecutorCompletionService<>(threadPoolExecutor);
public static void main(String[] args) throws InterruptedException, ExecutionException {
COMPLETION_SERVICE.submit(() -> {
sleep(300);
return 20;
});
COMPLETION_SERVICE.submit(() -> {
sleep(400);
return 10;
});
COMPLETION_SERVICE.submit(() -> {
sleep(100);
return 50;
});
for (int i = 0; i < 3; i++) {
Integer integer = COMPLETION_SERVICE.take().get();
threadPoolExecutor.execute(() -> System.out.println("将获取到的价格保存到数据库了:" + integer));
}
}
}
项目使用:
如上所以是为了处理批量任务等情况,项目上处理批量订单并返回处理结果(当然这里后续处理结果比较简单,整个模型更像是ThreadPoolExecutor#invokeAll),会有一个整体处理结果对象,里面有每一个订单会有一个处理结果。在处理时,我将每一个订单放入一个Callable中,代码如下:
public TransferResultDTO transferOrder(List<PurOrderBO> transferRequest) {
// 初始化结果容器
final TransferResultDTO resultDTO = new TransferResultDTO(Boolean.TRUE);
// 批量处理任务,一个PO对应一个线程
final Executor ttlExecutor = TtlExecutors.getTtlExecutor(THREAD_POOL_EXECUTOR_MAP.get(TRANSFER_ORDER.name()));
CompletionService<PoResultDTO> service = new ExecutorCompletionService<>(ttlExecutor, new LinkedBlockingDeque<>(transferRequest.size()));
// 拆单之后检查是否通过(只有在整体都通过的情况下,那么 业务账号,经销商,送达方是重复的),这三个检查都比较轻,先不考虑优化
transferRequest.stream().collect(Collectors.toMap(purOrderBO -> purOrderBO, purOrderBO -> service.submit(Objects.requireNonNull(TtlCallable.get(() -> {
// 初始化菜单后的返回结果
PoResultDTO poResultDTO = new PoResultDTO(purOrderBO.getPurOrderDTO().getPurOrderCode());
// 省略了比较复杂的订单处理流程。。。。
return poResultDTO;
}))))).forEach((key, value) -> {
try {
resultDTO.addPoResultDTO(service.take().get());
} catch (InterruptedException | ExecutionException e) {
// 执行线程被中断,可能是执行时间过程保护等, 需要稍后重试
log.error("addPoResultDTO = " + e);
}
});
return resultDTO;
}