自定义注解+面向切面（AOP）+oss+自定义线程池+CompletableFuture（多线程异步任务编排）实战，干货满满

引言

在作者上一篇博客中有提到，作者使用自己封装的导出文件方法来实现导出，在这一期中作者会详细解释自己导出方法使用过程；不清楚的各位看官可以先去查看作者的上一篇文章(上一篇文章链接），话不多说，直接上干货！

实战背景 

在项目开发过程中，多多少少都会遇到编写导出功能，像后台管理系统、电商平台、支付交易系统、财务系统，特别是报表系统里面的导出功能很多，然后每个开发者都会自己写一套，时间一长代码就容易混乱，严重影响阅读，也不规范，并且每次写大量重复的代码；为此作者在自己的实战项目中，自己编一套高可用，一次编码多处复用的方法供大家借鉴！

使用技术要点

使用到的技术点有：“自义定注解”、aop、oss、线程池、CompletableFuture和EasyExcel知识，不清楚的各位同学可以先行了解相关知识，然后再来阅读会更加流畅。

实战步骤

 1.引入pom

<dependencies>
    <dependency>
        <groupId>com.alibaba</groupId>
        <artifactId>easyexcel</artifactId>
        <version>最新版本号</version>
    </dependency>

     <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-test</artifactId>
            <version>5.3.26</version>
        </dependency>

 <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <version>1.18.26</version>
        </dependency>
        <dependency>
            <groupId>org.mapstruct</groupId>
            <artifactId>mapstruct</artifactId>
            <version>1.4.2.Final</version>
        </dependency>
        <dependency>
            <groupId>org.mapstruct</groupId>
            <artifactId>mapstruct-processor</artifactId>
            <version>1.4.2.Final</version>
        </dependency>
        <dependency>
            <groupId>io.swagger</groupId>
            <artifactId>swagger-annotations</artifactId>
            <version>1.6.6</version>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-core</artifactId>
            <version>5.3.26</version>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-context</artifactId>
            <version>5.3.26</version>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-tx</artifactId>
            <version>5.3.26</version>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-context-support</artifactId>
            <version>5.3.26</version>
        </dependency>

        <dependency>
            <groupId>jakarta.validation</groupId>
            <artifactId>jakarta.validation-api</artifactId>
            <version>2.0.2</version>
        </dependency>

        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-annotations</artifactId>
            <version>2.13.5</version>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-databind</artifactId>
            <version>2.13.5</version>
        </dependency>

        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>fastjson</artifactId>
            <version>1.2.83</version>
        </dependency>
</dependencies>

2.项目结构

项目的主要关注点就在两个地方一个是自定义注解（UploadFileToOSS）和UploadFileAspect类

3.抽取公共关键代码片段

  1.自定义注解UploadFileToOSS

    

import java.lang.annotation.*;

@Target(value={ElementType.METHOD,ElementType.TYPE,ElementType.FIELD,ElementType.CONSTRUCTOR,ElementType.PARAMETER})
@Retention(RetentionPolicy.RUNTIME)
@Documented
public @interface UploadFileToOSS {


	Class<?> value() ;

	String excelName() default "";


	String asynRecordSn() default "";
}

2.面向切面aop类UploadFileAspect

import com.alibaba.excel.EasyExcel;
import com.alibaba.excel.ExcelWriter;
import com.alibaba.excel.write.metadata.WriteSheet;
import com.alibaba.fastjson.JSON;
import com.wanhengtech.base.utils.BeanCopyUtil;
import lombok.extern.slf4j.Slf4j;
import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Pointcut;
import org.aspectj.lang.reflect.MethodSignature;
import org.springblade.common.annotation.UploadFileToOSS;
import org.springblade.common.endpoint.adapter.IAsynExportAdapter;
import org.springblade.common.upload.ExportProgressHandler;
import org.springblade.common.upload.dto.ExportTaskDispatchDTO;
import org.springblade.common.utils.CommonUtil;
import org.springblade.common.utils.ThreadLocalUtil;
import org.springblade.core.tool.api.R;
import org.springblade.core.tool.utils.Func;
import org.springblade.resource.feign.IOssClient;
import org.springframework.core.annotation.Order;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;
import org.springframework.stereotype.Component;
import org.springframework.web.multipart.MultipartFile;

import javax.annotation.Resource;
import java.io.IOException;
import java.lang.reflect.Method;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.CompletableFuture;
import java.util.stream.Collectors;
import java.util.stream.Stream;




@Aspect
@Component
@Slf4j
@Order(1000)
public class UploadFileAspect {

	@Resource(name = "exportAsyncThreadPoolExecutor")
	private ThreadPoolTaskExecutor taskExecutor;

     //自己项目定义的更新导出进度方法，不需要可以去掉
	@Resource
	private IAsynExportAdapter asynExportAdapter;

	@Resource
	private IOssClient ossClient;

	@Pointcut("@annotation(org.springblade.common.annotation.UploadFileToOSS)")
	public void uploadFileAspect() {
	}

	@Around("uploadFileAspect()")
	public Object beforePointcut(ProceedingJoinPoint joinPoint) throws Throwable {
        log.info("进入UploadFileToOSS切面--------异步导出----{}",joinPoint);
		//获取一个请求参数
		Object dtoObj = joinPoint.getArgs()[0];
		if (Func.isEmpty(dtoObj)){
			return R.fail("第一个必要参数不能空！");
		}
		//获取请求参数 应该list对象集合
		Object objectParam = joinPoint.getArgs()[1];
		log.info("ExportTaskDispatchDTO为:{}",JSON.toJSONString(dtoObj));
		ExportTaskDispatchDTO dto = BeanCopyUtil.beanCopy(dtoObj, ExportTaskDispatchDTO.class);
		MethodSignature signature = (MethodSignature) joinPoint.getSignature();
		Method method = signature.getMethod();
		UploadFileToOSS uploadFileToOSS = method.getDeclaredAnnotation(UploadFileToOSS.class);
		//得到传入的导出对象
		Class<?> classValue = uploadFileToOSS.value();
		//得到导出的文件名称
		StringBuffer buffer = new StringBuffer();
		//得到文件名称
		String fileName = buffer.append(dto.getFileName()).append(System.currentTimeMillis()).append(".xlsx").toString();
		log.info("得到文件名称为:{}",fileName);
		String asynRecordSn = dto.getTaskId();
		if (Func.isEmpty(asynRecordSn)){
			log.error("获取导出中心的任务id-------->asynRecordSn号失败！");
			return R.fail("获取导出中心的任务id为空！");
		}
		log.info("进入UploadFileToOSS切面--------异步导出线程----asynRecordSn号为:{}",asynRecordSn);
		//得到文件临时存储路径
		String filePath = CommonUtil.getFileSavePath() + fileName;
		log.info("得到文件临时存储路径为:{}",filePath);
		if (Func.isEmpty(objectParam) || (Func.isNotEmpty(objectParam) && Func.isEmpty(JSON.toJSONString(objectParam)))){
			log.error("得到导出的数据集合为空-------->asynRecordSn为：{}",asynRecordSn);
			asynExportAdapter.updateExportRecordProgressFail(asynRecordSn,"导出数据为空");
			return R.fail("获取导出的数据集合为空！");
		}
		List<?> resultList = JSON.parseArray(JSON.toJSONString(objectParam),classValue);
		//log.info("得到导出的数据集合为:{}",resultList);
		if (Func.isEmpty(resultList)){
			log.error("得到导出的数据集合为空-------->asynRecordSn为：{}",asynRecordSn);
			asynExportAdapter.updateExportRecordProgressFail(asynRecordSn,"数据条数为0");
			return R.fail("获取导出的数据集合为空！");
		}
		try {
			getAsyncUploadFile(uploadFileToOSS, classValue, fileName, asynRecordSn, filePath, resultList);
		}catch (Exception e){
			log.error("进入UploadFileToOSS切面--------异步导出异常:{}",e);
			asynExportAdapter.updateExportRecordProgressFail(asynRecordSn,e.getMessage());
			return R.fail("导出文件失败！");
		} finally {
			ThreadLocalUtil.remove();
		}




		//3、完成导出后返回
			Object result = joinPoint.proceed();
			return result;
	}
    /**
     * @Author Patrick
     * @Description  执行异步写入文件和上传文件至oss
     * @Date 2023-08-17 17:59
     **/
	private void getAsyncUploadFile(UploadFileToOSS uploadFileToOSS, Class<?> classValue, String fileName, String asynRecordSn, String filePath, List<?> resultList) throws InterruptedException, java.util.concurrent.ExecutionException {
		CompletableFuture<String> result = CompletableFuture.supplyAsync(() -> {
			ExcelWriter excelWriter = null;
			try {
				//设置本地线程序列号
				ThreadLocalUtil.set("asynRecordSn", asynRecordSn);

				List<List<?>> lists = new ArrayList<>();
				// xlsx 每个sheet最多 1048576 行，取1000000限制值
				Integer restrictedRange = 10000;
				// 一共生成多少sheet工作页
				Integer limit = (resultList.size() + restrictedRange - 1) / restrictedRange;
				log.info("excel分页数，limit:{}", limit);
				if (resultList.size() > restrictedRange) {
					lists = Stream
							.iterate(0, n -> n + 1)
							.limit(limit)
							.map(i -> resultList.stream().skip(i * restrictedRange).limit(restrictedRange).collect(Collectors.toList()))
							.collect(Collectors.toList());
				} else {
					lists.add(resultList);
				}
				//更新异步任务进度
				ExportProgressHandler handler = new ExportProgressHandler(fileName, resultList.size(), asynRecordSn, filePath, ossClient, asynExportAdapter);
				excelWriter = EasyExcel.write(filePath, classValue).registerWriteHandler(handler).build();
				for (int i = 0; i < lists.size(); i++) {
					// 每次都要创建writeSheet 这里注意必须指定sheetNo 而且sheetName必须不一样
					WriteSheet writeSheet = EasyExcel.writerSheet(i, Func.isNotBlank(uploadFileToOSS.excelName()) ? (uploadFileToOSS.excelName() + (i + 1)) : ("Sheet" + (i + 1))).build();
					excelWriter.write(lists.get(i), writeSheet);
				}
				return "ok";
			} catch (Exception e) {
				log.error("异步导出异常-写出文件到临时目录-------:{}", e);
				return "error";
			} finally {
				log.info("---开始----写完文件后清除导出的集合数据");
				resultList.clear();
				log.info("---结束----清除导出的集合数据");
				if (excelWriter!=null) {
					log.info("导出完成，关闭ExcelWriter，释放资源。");
					excelWriter.finish();
				}
			}
		}, taskExecutor);

		CompletableFuture<String> result2 = result.thenApply(rest -> {
			if ("ok".equals(rest)) {
				try {
					uploadFileOss(fileName, filePath, asynRecordSn);
				} catch (Exception e) {
					log.error("上传失败");
					throw new RuntimeException(e);
				} finally {
					log.info("清除临时文件,文件路径为:{}", filePath);
					boolean b = CommonUtil.deleteFile(filePath);
					if (b) {
						log.info("清除临时文件完成");
					} else {
						log.error("清除临时文件未完成{}", filePath);
					}
				}

				return "ok";
			}else {
				throw new RuntimeException("异步导出文件异常");
			}
		});

		if ("ok".equals(result2.get())){
			log.info("导出完成");
		}else {
			throw new RuntimeException("异步导出文件异常");
		}
	}

	private void uploadFileOss(String fileName,String tempFilePath,String asynRecordSn) throws IOException {
		log.info("fileName:{}, tempFilePath:{}, asynRecordSn:{}", fileName, tempFilePath, asynRecordSn);
		MultipartFile file = CommonUtil.convert(tempFilePath);
		log.info("asynRecordSn：{}，开始上传oss",asynRecordSn);
		R r = ossClient.putFileByNameSync(file, fileName);
		log.info("asynRecordSn：{}，上传oss返回为{}",asynRecordSn,r);
		if (!r.isSuccess()){
			log.error("远程调用oss,asynRecordSn为：{}上传oss失败！异常为：{}",asynRecordSn,r.getMsg());
			asynExportAdapter.updateExportRecordProgressFail(asynRecordSn,"上传oss失败");
			return;
		}
		if(Func.isNotEmpty(r.getData()) && r.isSuccess()){
			asynExportAdapter.uploadFileToOSSResultDownloadUrl(asynRecordSn,r.getData()+"",0);
			// 导出完成后，需删除临时文件
			log.info("导出完成");
		}else {
			log.error("远程调用oss,asynRecordSn为：{}，上传文件值oss返回下载链接为空！",asynRecordSn);

		}
	}

3.自定义线程池AsyncThreadPoolConfig

import lombok.extern.slf4j.Slf4j;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.annotation.EnableAsync;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;

import java.math.BigDecimal;
import java.util.concurrent.ThreadPoolExecutor;

/**
 * @Auther: Patrick
 * @Date: 2023-08-09 14:08
 * @Description: 异步线程池配置
 */
@Slf4j
@Configuration
@EnableAsync
public class AsyncThreadPoolConfig {


	@Bean(name = "exportAsyncThreadPoolExecutor")
	public ThreadPoolTaskExecutor taskExecutor() {
		//通过Runtime方法来获取当前服务器cpu内核，根据cpu内核来创建核心线程数和最大线程数
		int num = Runtime.getRuntime().availableProcessors();
		int availableProcessors = num < 2 ? 8:num;
		//最佳线程数目 = （（线程等待时间+线程CPU时间）/线程CPU时间 ）* CPU数目
		BigDecimal threadTime = BigDecimal.valueOf(1.5);
		BigDecimal cpuTime = BigDecimal.valueOf(0.5);
		int corePoolSize = threadTime.add(cpuTime).divide(cpuTime, 2, BigDecimal.ROUND_HALF_EVEN).multiply(BigDecimal.valueOf(availableProcessors)).intValue();
		int maxPoolSize = corePoolSize * 2;
		int queueCapacity = corePoolSize * 10;
		int keepAliveSeconds = 60;
		ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
		//核心线程数
		executor.setCorePoolSize(corePoolSize);
		//最大线程数
		executor.setMaxPoolSize(maxPoolSize);
		//队列最大长度
		executor.setQueueCapacity(queueCapacity);
		//最大空闲时间
		executor.setKeepAliveSeconds(keepAliveSeconds);
		executor.setThreadNamePrefix("taskExecutor-");
		//拒绝策略》CallerRunsPolicy()：交由调用方线程运行；AbortPolicy()：直接抛出异常；DiscardPolicy()：直接丢弃；DiscardOldestPolicy()：丢弃队列中最老的任务；
		executor.setRejectedExecutionHandler(new ThreadPoolExecutor.DiscardPolicy());
		log.info("异步线程池(taskExecutor)初始化完毕，cpu核数：{}，availableProcessors，核心线程数：{}，最大线程数：{}，队列最大长度：{}，最大空闲时间：{}", availableProcessors, corePoolSize, maxPoolSize, queueCapacity, keepAliveSeconds);
		return executor;
	}



}

4.实际项目使用截图

在需要使用导出公共代码的项目中应用上面抽取的公共pom坐标

1.在项目中先建一个bean类，专门用来触发aop切面上传操作。列如：

import c.w.o.a.model.*;
import lombok.extern.slf4j.Slf4j;
import org.springblade.common.annotation.UploadFileToOSS;
import org.springblade.common.upload.dto.ExportTaskDispatchDTO;
import org.springblade.core.tool.api.R;
import org.springframework.stereotype.Component;
/**
 * @Auther: Patrick
 * @Date: 2023-08-15 18:17
 * @Description: 触发自定义注解的切面逻辑bean
 */
@Slf4j
@Component
public class ExportFileBean {


	@UploadFileToOSS(value = MerchantOrderInfoExcel.class)
	public R getAsynExportMerchantOrderFile(ExportTaskDispatchDTO pageRequest, List<MerchantOrderInfoExcel> resultList){
		log.info("商户订单导出------调用公共的面向切面的异步导出文件方法");
		return R.success("调用公共的面向切面成功");
	}


	@UploadFileToOSS(value = MerchantEmailCardOrderInfoExcel.class)
	public R getAsynExportMerchantEmailCardOrderFile(ExportTaskDispatchDTO pageRequest, List<MerchantEmailCardOrderInfoExcel> resultList){
		log.info("商户邮件卡密订单导出------调用公共的面向切面的异步导出文件方法");
		return R.success("调用公共的面向切面成功");
	}


}

2.在需要使用导出功能的Service层中注入上面的bean，例如：

完整代码在上一篇文章的 “拓展实际应用” 目录下可以查看

5.复用说明

其他项目或者模块都可以按照上面的步骤重复操作即可，单体项目只需要创建一个触发bean，微服务项目需要在每个服务中创建一个触发bean，然后就可以统一使用上面的公共代码进行上传和导出文件了。是不是很方便！

6.结尾术语

个人经验，不喜勿喷，欢迎大家提供新的优化点，欢迎在评论区讨论！