昨天线上的导出文件数据出现了问题(导出的数据= A表通过sql查出来+B表通过异步接口查询出来数据拼装),经过排查是由于我使用了Spring core包中的ThreadPoolTaskExecutor类去异步获取另一个B表中的数据,B表中的数据还没有完全返回回来。接口就已经返回了,所以导致导出数据问题。
1. 导出数据会丢失的代码如下:
<!-- 通用异步执行器 -->
<bean id="taskExecutor"
class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<property name="corePoolSize" value="10" />
<property name="maxPoolSize" value="30" />
</bean>
@Autowired
TaskExecutor taskExecutor;
List<OrderDto> orderDtoList = new ArrayList<>();
List<Order> orderList = OrderService.selectOrderList(query);
for (Order brokerOrder : orderList) {
OrderDto orderDto = new OrderDto();
BeanUtils.copyProperties(brokerOrder, orderDto);
taskExecutor.execute(new Runnable() {
@Override
public void run() {
try {
// 查计划信息
AmountQuery amountQuery = new AmountQuery();
amountQuery.setOrderId(brokerOrder.getOrderId());
amountQuery.setOrderCode(brokerOrder.getOrderCode());
Return<Result<AmountDto>> amountList = amountFacadeService.selectOrderPayAmountList(amountQuery);
Result pageResult = amountList.R;
if (pageResult != null && pageResult.getData() != null && pageResult.getData().size() > 0) {
List<AmountDto> alist = new ArrayList<>();
List<AmountDto> list = pageResult.getData();
for (AmountDto dto : list) {
alist.add(dto);
}
orderDto.setAmountList(alist);
}
} catch (Exception e) {
LOG.info("error ",e);
}
}
});
orderDtoList.add(orderDto);
}
int count = OrderService.selectOrderListCount(query);
return Return.create(new Result<OrderDto>(count, orderDtoList));
2. 然后换成了多线程的Futrue后,问题解决,部分代码如下:
private final ExecutorService es = new ThreadPoolExecutor(8, 10, 100, TimeUnit.MILLISECONDS, new LinkedBlockingQueue<Runnable>(5000));
List<TruckBrokerOrderDto> orderDtoList = new ArrayList<>();
List<order> orderList = orderService.selectOrderList(query);
Map<String, Future<orderDto>> futureMap = new HashMap<>();
for (Order order : orderList) {
Future<OrderDto> orderFuture = es.submit(new Callable<OrderDto>() {
@Override
public OrderDto call() {
OrderDto orderAmountDto = new OrderDto();
// 查计划信息
AmountQuery amountQuery = new AmountQuery();
amountQuery.setOrderId(order.getOrderId());
amountQuery.setOrderCode(order.getOrderCode());
List<Amount> amountList = amountService.selectAmountList(amountQuery);
if (amountList != null && amountList.size() > 0) {
// 数据处理
}
return orderAmountDto;
}
});
futureMap.put(order.getOrderCode(), orderFuture);
}
for (Order order : orderList) {
OrderDto orderDto = new OrderDto();
BeanUtils.copyProperties(brokerOrder, orderDto);
try {
OrderDto orderAmountDto = futureMap.get(order.getOrderCode()).get();
if (orderAmountDto.getAmountList() != null && orderAmountDto.getAmountList().size() > 0) {
orderDto.setAmountList(orderAmountDto.getAmountList());
}
} catch (Exception e) {
LOG.info("error " ,e);
}
orderDtoList.add(orderDto);
}
int count = OrderService.selectOrderListCount(query);
returnReturn.create(new Result<OrderDto>(count, orderDtoList));
总结:
一般不需要对异步返回数据做处理的话,可以直接用Spring core包中的ThreadPoolTaskExecutor类直接执行(会丢失数据),比如发送短信,异步更新数据不需要返回结果等。
假如要对异步返回数据做处理的话,请用多线程的Futrue模式,可以保证数据不丢失。