需求:有这样一个需求,从数据库捞一批dsl,进行查询,看dsl是否正确,如果一个一个跑非常慢,使用Java多线程去跑,加速查询速度
思路:使用java线程池,设置好线程池的参数,创建好线程池,提交这些任务到线程池,返回值是一个Future,我们把所有Future放到一个List里,所有任务提交完,我们再遍历List<Future>,一个一个get结果即可
具体代码:
1. 查询任务代码:
public class QueryTask implements Callable {
private final String dsl;
private final QueryService queryService;
public QueryTask(QueryService queryService, String dsl) {
this.dsl = dsl;
this.queryService = queryService;
}
@Override
public Object call() throws Exception {
QueryResult queryResult = queryService.doQuery(dsl);
log.info("queryResult is {}", JSON.toJSONString(queryResult));
if (queryResult.getCode() != 200) {
return dsl;
}
if(queryResult.getCode() ==200){
List<DataResult> result = queryResult.getResult();
for (DataResult dataResult : result) {
if(dataResult.getResultStatus()!=SUCCESS){
return dsl;
}
}
}
return "";
}
}
2.导出接口代码:
@RequestMapping(value = "/export-multithreading", method = RequestMethod.GET)
public void exportPhones(HttpServletResponse response) throws IOException{
StopWatch stopWatch = new StopWatch();
stopWatch.start();
//1.创建一个线城池
ExecutorService taskPool = new ThreadPoolExecutor(16,32,200L, TimeUnit.MILLISECONDS,new LinkedBlockingQueue(5000),
new ThreadFactoryBuilder().setNameFormat("thread-thirdPushMsgJob-runner-%d").build());
//2.从数据获取一批dsl
List<String> dslList = scriptService.batchBearDslToFinderDsl("event", new ArrayList<>());
Integer removeNumber = 0;
Integer totalNumber = dslList.size();
//3.提交任务
List<Future> futureList = new ArrayList<>();
Iterator it = dslList.iterator();
while(it.hasNext()){
String dsl = (String) it.next();
try {
QueryTask aTask = new QueryTask(queryService,dsl);
Future<String> submit = taskPool.submit(aTask);
futureList.add(submit);
}catch (Exception e){
log.error("query failed {}",dsl);
it.remove();
}
}
//4.处理结果
for (Future future : futureList) {
try {
String result = (String) future.get();
if (!"".equals(result)) {
dslList.remove(result);
removeNumber++;
}
}catch (Exception e){
log.error("future failed:{}",e.getMessage());
}
}
log.info("dslList size:{},remove size:{}",dslList.size(),removeNumber);
stopWatch.stop();
dslList.add("useAble number:"+dslList.size()+"total number:"+totalNumber+"remove number:"+removeNumber);
dslList.add(stopWatch.prettyPrint());
//5.导出文件
response.setContentType("text/plain");
String fileName = System.currentTimeMillis()+"";
response.setHeader("Content-Disposition", "attachment; filename=" + fileName + ".txt");
BufferedOutputStream buff = null;
StringBuilder write = new StringBuilder();
String enter = "\r\n";
ServletOutputStream outSTr = null;
try {
outSTr = response.getOutputStream(); // 建立
buff = new BufferedOutputStream(outSTr);
// 把内容写入文件
if (dslList.size() > 0) {
for (String s : dslList) {
write.append(s);
write.append(enter);
write.append(enter);
}
}
buff.write(write.toString().getBytes(StandardCharsets.UTF_8));
buff.flush();
buff.close();
} catch (Exception e) {
log.error("export txt failed:",e);
} finally {
assert buff != null;
buff.close();
outSTr.close();
}
}
我这是以批量提交dsl进行查询为列,其他多线程操作类似