1.1、定义线程池
public class ThreadPoolUtils {
private static int corePoolSize = 50;
private static int maximumPoolSize = 100;
private static long keepAliveTime = 50L;
private static int workQueueSize = 3000;
private static ExecutorService executor;
static {
executor = new ThreadPoolExecutor(corePoolSize, maximumPoolSize, keepAliveTime, TimeUnit.SECONDS,
new ArrayBlockingQueue<>(workQueueSize), (task, executor) -> log.info("ThreadPoolUtils process business is full"));
}
public static void submit(Runnable task) {
executor.submit(task);
}
//当前核心数两倍的线程池
private static final int PROCESSORS = Runtime.getRuntime().availableProcessors() << 1;
public static final ThreadPoolExecutor DOUBLE_PROCESSORS_THREAD_POOL = new ThreadPoolExecutor(
PROCESSORS,
PROCESSORS,
20,
TimeUnit.SECONDS,
new ArrayBlockingQueue<>(1000),
r -> {
Thread thread = new Thread(r);
thread.setName("PROCESSORS_DEFAULT_THREAD_POOL-" + r.getClass().getName());
return thread;
},
new ThreadPoolExecutor.CallerRunsPolicy());
}
1.2、使用线程池分片查询数据库
List<String> inputList; // 查询条件,根据具体场景定义
List<String> res;
long start = System.currentTimeMillis();
int groupSize = 300;
// 分成 groupSize 个一组
// getData 是数据库查询的方法,inputList 为入参,返回结果为 List<String>
List<List<String>> results = IntStream.range(0, inputList.size() / groupSize +1)
.mapToObj(i -> CompletableFuture.supplyAsync(() ->
getData(inputList.subList(i * groupSize,Math.min((i + 1) * groupSize,inputList.size()) ))
, ThreadPoolUtils.DOUBLE_PROCESSORS_THREAD_POOL))
.map(CompletableFuture::join)
.collect(Collectors.toList());
// 处理结果达到预期格式
res = results.stream()
.flatMap(List::stream)
.collect(Collectors.toList());
log.info("线程池分片查询sql耗时:"+ (System.currentTimeMillis() - start) + "ms");
实际应用场景可根据根据 groupSize 自定义切片大小,寻找一个最优解。