多线程并行处理数据

多线程并行处理数据

在工作中很多时候会因为各种原因选择使用多线程并行去处理我们要处理的数据，前段时间工作中就遇到了需要多线程去处理的业务逻辑情况，记录一下。

业务场景

在后台系统代码中，首先会拿到一些条件，然后会根据入参组合成一个modelList，需要用modelList去循环远程调用第三方的系统，调用第三方系统执行速度很慢，调用一次需要1至2秒，modelList稍微长一点的时候就会出现循环调用占用时间太长的问题。

解决方法

使用多线程去并行调用第三方，多个接口同时调用，减少调用时间

部分逻辑代码

    @Autowired(required = false)
    private JedisPool jedisPool;

    @Resource
    private ThreadPoolExecutor threadPoolExecutor;
    
	@Override
    public List<HisAppointmentSourceDTO> getAppointmentSourceByCondition(HisAppointmentSourceModel hisAppointmentSourceModel) {
        try {
            //定义调用his的modelList
            List<HisDocApptSourceModel> modelList = Lists.newArrayList();
            HisDocApptSourceModel hisDocApptSourceModel = HisDocApptSourceModel.builder().build();
            Integer latchSize = 0;
            //入参方式1: 医生部门Map,开始时间,结束时间
            if (null != hisAppointmentSourceModel.getDocMap() && !hisAppointmentSourceModel.getDocMap().isEmpty()){
                latchSize = hisAppointmentSourceModel.getDocMap().size();
                for (String docCode : hisAppointmentSourceModel.getDocMap().keySet()) {
                    hisDocApptSourceModel = HisDocApptSourceModel.builder().startDate(DateUtils.format(hisAppointmentSourceModel.getStartDate(),CommonConstant.YYYY_MM_DD)).endDate(DateUtils.format(hisAppointmentSourceModel.getEndDate(),CommonConstant.YYYY_MM_DD)).docCode(docCode).deptCode(hisAppointmentSourceModel.getDocMap().get(docCode)).build();
                    modelList.add(hisDocApptSourceModel);
                }
            }else if (!StringUtils.isEmpty(hisAppointmentSourceModel.getDeptCode())){
                if (!CollectionUtils.isEmpty(hisAppointmentSourceModel.getDoctorCodeList())){
                    //入参方式2: 部门code,医生codeList,开始时间,结束时间
                    latchSize = hisAppointmentSourceModel.getDoctorCodeList().size();
                    for (String docCode : hisAppointmentSourceModel.getDoctorCodeList()) {
                        hisDocApptSourceModel = HisDocApptSourceModel.builder().startDate(DateUtils.format(hisAppointmentSourceModel.getStartDate(),CommonConstant.YYYY_MM_DD)).endDate(DateUtils.format(hisAppointmentSourceModel.getEndDate(),CommonConstant.YYYY_MM_DD)).docCode(docCode).deptCode(hisAppointmentSourceModel.getDeptCode()).build();
                        modelList.add(hisDocApptSourceModel); }
                }else {
                    //入参方式3: 部门code,开始时间,结束时间
                    //查询该段时间的该部门下所有出诊医生
                    HisApptDocInfoModel model = HisApptDocInfoModel.builder().deptCode(hisAppointmentSourceModel.getDeptCode()).startDate(DateUtils.format(hisAppointmentSourceModel.getStartDate(),CommonConstant.YYYY_MM_DD)).endDate(DateUtils.format(hisAppointmentSourceModel.getEndDate(),CommonConstant.YYYY_MM_DD)).build();
                    List<HisApptDocInfoDTO> hisApptDocInfoDTOList = getApptDocInfo(model);
                    latchSize = hisApptDocInfoDTOList.size();
                    for (HisApptDocInfoDTO hisApptDocInfoDTO : hisApptDocInfoDTOList) {
                        hisDocApptSourceModel = HisDocApptSourceModel.builder().startDate(DateUtils.format(hisAppointmentSourceModel.getStartDate(),CommonConstant.YYYY_MM_DD)).endDate(DateUtils.format(hisAppointmentSourceModel.getEndDate(),CommonConstant.YYYY_MM_DD)).docCode(hisApptDocInfoDTO.getYsdm()).deptCode(hisAppointmentSourceModel.getDeptCode()).build();
                        modelList.add(hisDocApptSourceModel);
                    }
                }
            }
            CountDownLatch latch = new CountDownLatch(latchSize);
            List<CompletableFuture<List<AppointmentSourceDO>>> futures = Lists.newArrayList();
            List<AppointmentSourceDO> appointmentSourceDOList = Lists.newArrayList();
            //多线程并行调用his
            for (HisDocApptSourceModel model: modelList) {
                CompletableFuture<List<AppointmentSourceDO>> future = CompletableFuture.supplyAsync(() -> {
                    //调用his查询医生号源
                    List<HisDocApptSourceResponse> docApptSourceResponseList = hisServiceAdapter.getDocApptSource(model);
                    //筛选对应部门号源
                    List<AppointmentSourceDO> list = Lists.newArrayList();
                    for (HisDocApptSourceResponse hisDocApptSourceResponse : docApptSourceResponseList) {
                        if (hisDocApptSourceResponse.getKsdm().equals(model.getDeptCode())) {
                            AppointmentSourceDO appointmentSourceDO = buildeCommonDo(hisDocApptSourceResponse);
                            appointmentSourceDO.setDoctorCode(model.getDocCode());
                            list.add(appointmentSourceDO);
                        }
                    }
                    latch.countDown();
                    return list;
                }, threadPoolExecutor);
                futures.add(future);
            }
            latch.await(5, TimeUnit.SECONDS);
            // 再次汇总全量信息
            List<List<AppointmentSourceDO>> taskResults = futures.stream().map(CompletableFuture::join).collect(Collectors.toList());
            appointmentSourceDOList = taskResults.stream().flatMap(Collection::stream).collect(Collectors.toList());

            if (!CollectionUtils.isEmpty(appointmentSourceDOList)) {
                return appointmentSourceDOList.stream().map(appointmentSourceDO -> {
                    HisAppointmentSourceDTO hisAppointmentSourceDTO = HisAppointmentSourceDTO.builder().build();
                    BeanUtils.copyProperties(appointmentSourceDO, hisAppointmentSourceDTO);
                    return hisAppointmentSourceDTO;
                }).collect(Collectors.toList());
            } else {
                return new ArrayList<>();
            }
        }catch (Exception e){
            log.error("查询医生号源信息异常", e);
            return new ArrayList<>();
        }
    }

回顾

1：先根据各种入参方式拼装modelList。
2：创建泛型为CompletableFuture的List ：futures ，用于存放每个线程执行拿到的结果。
3：循环modelList 创建CompletableFuture对象 future，拿到执行结果后添加到futures 中。
4：用futures汇总全部信息，做对象转换操作。
5：返回结果集。