使用SpringTask+数据库+Redis,自定义切面对不同入参的接口数据进行Redis缓存,并根据不同频率进行刷新

该方案提出了一种接口缓存策略,利用SpringAOP和Redis实现接口数据的缓存。通过注解定义超时时间和更新cron表达式,当接口被调用时,数据会被存储到Redis中。同时,通过前端Header参数判断接口调用方式,更新接口活跃时间,并根据活跃时间管理缓存更新。此外,使用MySQL存储接口信息,通过线程池和定时任务对不同接口进行不同频率的更新,以及在服务关闭时优雅地停止定时任务。
摘要由CSDN通过智能技术生成

概要

  • 本方案是由于工作中出现的系统数据库某些表数据量大(几千万)、关联表多等业务情况,导致接口响应速度过长,同时业务接口数量也过多,故而需要个方案对部分旧接口进行改造,所以自己便在工作之余设想的一个较为通用的接口缓存方案
  • 本方案主要采用了注解的方式,利用Spring的Aop特性,通过把接口放回数据更新进Redis中,实现接口数据的缓存
  • 同时采用数据库(案例是mysql),对接口的类名、方法、入参、更新cron等进行存储,采用反序列化的方式来实现接口的调用,并通过SpringTask对不同的接口进行不同频率的定时更新
  • 采用前端Headers添加isActive入参的方式(试过其他的方式但需要适配),判断接口是否是接口请求方式调用,当是接口请求方式调用时会更新表字段last_active_time,根据该时间可以对长期没被主动调用过的接口进行停止缓存更新,一定程度上防止系统的内存占用越来越大

 导入依赖:

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
            <exclusions>
                <exclusion>
                    <groupId>org.springframework.boot</groupId>
                    <artifactId>spring-boot-starter-logging</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-aop</artifactId>
            <version>3.0.4</version>
        </dependency>

        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
        </dependency>

        <dependency>
            <groupId>com.baomidou</groupId>
            <artifactId>mybatis-plus-boot-starter</artifactId>
            <version>3.5.3.1</version>
        </dependency>

        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-log4j2</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-redis</artifactId>
        </dependency>

        <dependency>
            <groupId>com.google.guava</groupId>
            <artifactId>guava</artifactId>
            <version>31.1-jre</version>
        </dependency>

        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>fastjson</artifactId>
            <version>2.0.32</version>
        </dependency>

类介绍  

  • RedisCache: 自定义接口缓存注解
    @Retention(RetentionPolicy.RUNTIME)
    @Target(ElementType.METHOD)
    public @interface RedisCache {
    
        /**
         * 超时时间
         * */
        int timeout() default 61*3;
    
        /**
         * 超时时间单位
         * */
        TimeUnit timeUnit() default TimeUnit.SECONDS;
    
        /**
         * 自定义cron
         * */
        String cron() default "*/3 * * * * *";
    
    }

  • RedisCacheAspect: 自定义接口缓存注解RedisCache的切面
    /**
     * @author yuqingxin
     * 自定义接口缓存注解RedisCache的切面
     * */
    @Aspect
    @Component
    @Log4j2
    public class RedisCacheAspect {
    
        final RedisCacheService redisCacheService;
    
        final Executor threadPool;
    
        final ApiActiveService apiActiveService;
    
        final ApiScheduleMapper apiScheduleMapper;
    
        public RedisCacheAspect(RedisCacheService redisCacheService,
                                Executor threadPool,
                                ApiActiveService apiActiveService,
                                ApiScheduleMapper apiScheduleMapper){
            this.redisCacheService = redisCacheService;
            this.threadPool = threadPool;
            this.apiActiveService = apiActiveService;
            this.apiScheduleMapper = apiScheduleMapper;
        }
    
        private final ObjectMapper objectMapper = new ObjectMapper();
    
        @Around("@annotation(redisCache)")
        public Object getControllerData(ProceedingJoinPoint joinPoint, RedisCache redisCache) throws Throwable {
             //采用前端Header入参方式判断调用该方法的方式
            ServletRequestAttributes attributes = (ServletRequestAttributes) RequestContextHolder.getRequestAttributes();
            String isActive = null;
            if (attributes != null) {
                HttpServletRequest request = attributes.getRequest();
                isActive = request.getHeader("isActive");
            }
    
            //获取方法的相关信息
            String className = joinPoint.getTarget().getClass().getName();
            String arguments = objectMapper.writeValueAsString(joinPoint.getArgs());
            Method method = ((MethodSignature) joinPoint.getSignature()).getMethod();
            String methodName = method.getName();
            String argumentsType = Arrays.toString(method.getParameterTypes());
    
            // 构建缓存键:类名 + 方法 + 参数值
            String key = className + ":" + methodName + ":" + arguments;
            String result = redisCacheService.get(key);
            log.info("RedisCache:{}",result);
            if (result == null) {
                //调用接口逻辑并获取对应的返回数据并转换为字符串
                result =  objectMapper.writeValueAsString(joinPoint.proceed());
                log.info("set RedisCache:{} :: {}",key,result);
                redisCacheService.set(key, result, redisCache.timeout(),redisCache.timeUnit());
            }
    
            //定时调用->更新redis数据
            if (null == isActive){
                threadPool.execute(()-> {
                    try {
                        String updateResult = objectMapper.writeValueAsString(joinPoint.proceed());
                        log.info("Set RedisCache:{} :: {}", key, updateResult);
                        redisCacheService.set(key, updateResult, redisCache.timeout(), redisCache.timeUnit());
                    } catch (Throwable e) {
                        log.error("Set RedisCache Error:{}", key);
                        e.printStackTrace();
                    }
                });
            }
    
            //把记录更新到api缓存表
            String finalIsActive = isActive;
            threadPool.execute(()->{
                LocalDateTime now = LocalDateTime.now();
                LambdaQueryWrapper<ApiScheduleEntity> asL = new LambdaQueryWrapper<>();
                asL.eq(ApiScheduleEntity::getClassName,className)
                        .eq(ApiScheduleEntity::getMethodName,method.getName())
                        .eq(ApiScheduleEntity::getArgumentsType,argumentsType)
                        .eq(ApiScheduleEntity::getArguments,arguments);
                ApiScheduleEntity apiSchedule = apiScheduleMapper.selectOne(asL);
                if (apiSchedule != null){
                    apiSchedule.setUpdateAt(now);
                    //主动调用 则 更新 主动调用时间
                    if ("1".equals(finalIsActive)){
                        apiSchedule.setLastActiveTime(now);
                    }if (1 != apiSchedule.getStatus()){
                        apiSchedule.setStatus(1);
                        //添加对应的定时任务
                        log.info("addApiScheduledTask:{}"+key);
                        apiActiveService.addApiScheduledTask(apiSchedule);
                    }
                    apiScheduleMapper.updateById(apiSchedule);
                }else {
                    ApiScheduleEntity apiScheduleEntity = new ApiScheduleEntity();
                    apiScheduleEntity.setClassName(className);
                    apiScheduleEntity.setMethodName(method.getName());
                    apiScheduleEntity.setArgumentsType(argumentsType);
                    apiScheduleEntity.setArguments(arguments);
                    apiScheduleEntity.setCron(redisCache.cron());
                    apiScheduleEntity.setLastActiveTime(now);
                    apiScheduleEntity.setUpdateAt(now);
                    apiScheduleMapper.insert(apiScheduleEntity);
    
                    //添加对应的定时任务
                    log.info("addApiScheduledTask:{}"+key);
                    apiActiveService.addApiScheduledTask(apiScheduleEntity);
                }
            });
            //把获取到的字符串转化为接口的返回类型
            return objectMapper.readValue(result,method.getReturnType());
        }
    }

  • RedisCacheService: RedisTemplate<String,String>的部分简单封装
    @Service
    public class RedisCacheService {
    
        private final RedisTemplate<String,String> redisTemplate;
    
        public RedisCacheService(RedisTemplate<String,String> redisTemplate){
            this.redisTemplate = redisTemplate;
        }
    
        public String get(String key){
            return redisTemplate.opsForValue().get(key);
        }
    
        public void set(String key, String value){
            redisTemplate.opsForValue().set(key,value,60, TimeUnit.SECONDS);
        }
    
        public void set(String key, String value, long seconds){
            redisTemplate.opsForValue().set(key,value,seconds, TimeUnit.SECONDS);
        }
    
        public void set(String key, String value, long seconds, TimeUnit timeUnit){
            redisTemplate.opsForValue().set(key,value,seconds, timeUnit);
        }
    }

  • RedisScheduleConfig:自定义定时任务配置,主要作用为在服务启动时添加添加需要缓存的接口的定时任务
    /**
     * @author yuqingxin
     * */
    @Log4j2
    @Configuration
    @EnableScheduling
    public class RedisScheduleConfig implements SchedulingConfigurer {
    
        final Executor threadPool;
    
        final ApplicationContext applicationContext;
    
        final ApiActiveService apiActiveService;
    
        final ThreadPoolTaskScheduler threadPoolTaskScheduler;
    
        final Map<String,ScheduledFuture<?>> scheduledFutureMap;
    
        public RedisScheduleConfig(Executor threadPool,
                                   ApplicationContext applicationContext,
                                   ApiActiveService apiActiveService,
                                   ThreadPoolTaskScheduler threadPoolTaskScheduler,
                                   Map<String,ScheduledFuture<?>> scheduledFutureMap){
            this.threadPool = threadPool;
            this.applicationContext = applicationContext;
            this.apiActiveService = apiActiveService;
            this.threadPoolTaskScheduler = threadPoolTaskScheduler;
            this.scheduledFutureMap = scheduledFutureMap;
        }
    
        /**
         * 程序启动时添加定时任务
         * 如果是分布式服务,一般只用一台服务器更新接口缓存就行
         * 可以简单通过设置服务器/容器的环境变量,使用@Value获取变量值来判断该服务器/容器是否是启用更新接口缓存
         * */
        @Override
        public void configureTasks(@Nonnull ScheduledTaskRegistrar scheduledTaskRegistrar) {
            //查询所有需要更新缓存的接口的定时任务
            log.info("初始化需缓存接口的定时任务");
            List<ApiScheduleEntity> apiScheduleList = apiActiveService.getApiScheduleList();
            for (ApiScheduleEntity apiSchedule : apiScheduleList) {
                try {
                    apiActiveService.addApiScheduledTask(apiSchedule);
                }catch (Exception e){
                    log.error("Init Active Api Error:{}",apiSchedule);
                }
            }
        }
    
    }

  • ApplicationClosedListenerConfig:自定义监听服务关闭配置,主要目的为在服务关闭时先关闭正在运行的定时任务
    @Configuration
    public class ApplicationClosedListenerConfig implements ApplicationListener<ContextClosedEvent> {
    
        final ThreadPoolTaskScheduler threadPoolTaskScheduler;
    
        public ApplicationClosedListenerConfig(ThreadPoolTaskScheduler threadPoolTaskScheduler){
            this.threadPoolTaskScheduler = threadPoolTaskScheduler;
        }
    
        @Override
        public void onApplicationEvent(@Nonnull ContextClosedEvent contextClosedEvent) {
            threadPoolTaskScheduler.shutdown();
        }
    }

  • ThreadPoolConfig: 线程池相关配置,包括线程池、线程池任务调度器、定时任务类缓存
    
    @Log4j2
    @Configuration
    public class ThreadPoolConfig {
    
        @Value("${thread-pool.core-pool-size}")
        private int corePoolSize;
    
        @Value("${thread-pool.max-pool-size}")
        private int maxPoolSize;
    
        @Value("${thread-pool.keep-alive-time}")
        private long keepAliveTime;
    
        @Bean
        public Executor threadPool() {
            log.info("创建自定义线程池");
            return new ThreadPoolExecutor(
                    corePoolSize,
                    maxPoolSize,
                    keepAliveTime,
                    TimeUnit.SECONDS,
                    new LinkedBlockingQueue<>(1000),
                    new ThreadFactoryBuilder().setNameFormat("Thread-pool-%d").build(),
                    new ThreadPoolExecutor.AbortPolicy()
            );
        }
    
        @Bean
        public ThreadPoolTaskScheduler threadPoolTaskScheduler(){
            log.info("创建自定义线程池任务调度器");
            ThreadPoolTaskScheduler threadPoolTaskScheduler = new ThreadPoolTaskScheduler();
            threadPoolTaskScheduler.setPoolSize(20);
            threadPoolTaskScheduler.setThreadGroupName("TaskScheduler-");
            threadPoolTaskScheduler.setAwaitTerminationSeconds(60);
            threadPoolTaskScheduler.setWaitForTasksToCompleteOnShutdown(true);
            return threadPoolTaskScheduler;
        }
    
        @Bean
        public Map<String, ScheduledFuture<?>> scheduledFutureMap(){
            log.info("创建定时任务缓存集合");
            return new HashMap<>();
        }
    }

  • ArgumentsUtil:实现参数更对应类的转换方法类
    /**
     * @author yuqingxin
     * */
    @Service
    public class ArgumentsUtil {
    
        final ObjectMapper objectMapper;
    
        public ArgumentsUtil(ObjectMapper objectMapper){
            this.objectMapper = objectMapper;
        }
    
        /**
         * 参数类型转换为对应对象
         * */
        public Class<?>[] getClassArray(String typesString){
            String[] classNames = typesString.substring(1, typesString.length() - 1).split("class ");
            Class<?>[] classes = new Class<?>[classNames.length-1];
    
            for (int i = 1; i < classNames.length; i++) {
                try {
                    Class<?> clazz = Class.forName(classNames[i].replace(", ",""));
                    classes[i-1] = clazz;
                } catch (ClassNotFoundException e) {
                    e.printStackTrace();
                }
            }
            return classes;
        }
    
        /**
         * 参数值转换为对象
         * */
        public Object[] getObjectArray(String argumentsString,Class<?>[] parameterTypes) {
            Object[] objects = JSON.parseObject(argumentsString,Object[].class);
            for (int i = 0; i < objects.length; i++) {
                try {
                    objects[i] = objectMapper.readValue(objectMapper.writeValueAsString(objects[i]),parameterTypes[i]);
                } catch (JsonProcessingException e) {
                    throw new RuntimeException(e);
                }
            }
            return objects;
        }
    }

  • ApiScheduleEntity: 接口缓存调用表对应的实体类
    /**
     * @author yuqingxin
     * */
    @Data
    @AllArgsConstructor
    @NoArgsConstructor
    @TableName("tbl_api_schedule")
    public class ApiScheduleEntity {
    
        private Long id;
    
        /**
         * 类名
         * */
        private String className;
    
        /**
         * 方法名
         * */
        private String methodName;
    
        /**
         * 参数类型
         * */
        private String argumentsType;
    
        /**
         * 参数值
         * */
        private String arguments;
    
        /**
         * 更新频率cron
         * */
        private String cron;
    
        /**
         * 最后调用时间
         * */
        private LocalDateTime updateAt;
    
        /**
         * 最后通过接口调用方式的调用时间
         * */
        private LocalDateTime lastActiveTime;
    
        /**
         * 是否启用:1是0否
         * */
        private Integer status;
    }
    

  • ApiActiveService: 调用缓存接口相关逻辑类,主要功能为:
    • 获取需要调用/长时间为调用的缓存接口
    • 通过反序列的方式对单个/多个需缓存的接口进行调用
    • 对长期未使用的缓存接口取消对应的定时任务
      /**
       * @author yuqingxin
       * */
      @Log4j2
      @Service
      public class ApiActiveService {
      
          //超时时间
          private final Integer timeout = 60*60*24*3;
      
          final ApiScheduleMapper apiScheduleMapper;
      
          final ApplicationContext applicationContext;
      
          final ThreadPoolTaskScheduler threadPoolTaskScheduler;
      
          final Executor threadPool;
      
          final Map<String, ScheduledFuture<?>> scheduledFutureMap;
      
          final ArgumentsUtil argumentsUtil;
      
          public ApiActiveService(ApiScheduleMapper apiScheduleMapper,
                                  ApplicationContext applicationContext,
                                  ThreadPoolTaskScheduler threadPoolTaskScheduler,
                                  Executor threadPool,
                                  Map<String, ScheduledFuture<?>> scheduledFutureMap,
                                  ArgumentsUtil argumentsUtil){
              this.apiScheduleMapper = apiScheduleMapper;
              this.applicationContext = applicationContext;
              this.threadPoolTaskScheduler = threadPoolTaskScheduler;
              this.threadPool = threadPool;
              this.scheduledFutureMap = scheduledFutureMap;
              this.argumentsUtil = argumentsUtil;
          }
      
          /**
           * 获取需要更新的Api
           * */
          public List<ApiScheduleEntity> getApiScheduleList(){
              LambdaQueryWrapper<ApiScheduleEntity> aL =  new LambdaQueryWrapper<>();
              aL.eq(ApiScheduleEntity::getStatus,1);
              return apiScheduleMapper.selectList(aL);
          }
      
          /**
           * 获取长时间未使用的Api
           * @param timeout 多长时间未使用  秒
           * */
          public List<ApiScheduleEntity> getLongTimeUnUseApiList(Integer timeout){
              LambdaQueryWrapper<ApiScheduleEntity> aL =  new LambdaQueryWrapper<>();
              aL.eq(ApiScheduleEntity::getStatus,1)
                      .le(ApiScheduleEntity::getLastActiveTime, LocalDateTime.now().minusSeconds(timeout));
              return apiScheduleMapper.selectList(aL);
          }
      
          /**
           * 调用单个Api方法
           * */
          public void  runApiOne(ApiScheduleEntity apiSchedule){
              // 类名
              String className = apiSchedule.getClassName();
              // 方法名
              String methodName = apiSchedule.getMethodName();
              // 参数值
              String argumentsString = apiSchedule.getArguments();
      
              // 参数类型
              String argumentsType = apiSchedule.getArgumentsType();
              Class<?>[] parameterTypes = argumentsUtil.getClassArray(argumentsType);
              Object[] arguments = argumentsUtil.getObjectArray(argumentsString,parameterTypes);
              try {
                  // 获取类对象
                  Class<?> apiClass = Class.forName(className);
                  // 获取方法对象
                  Method method = apiClass.getMethod(methodName, parameterTypes);
                  // 获取该类的Bean对象
                  Object instance = applicationContext.getBean(apiClass);
                  // 调用方法
                  Object result = method.invoke(instance, arguments);
                  // 处理方法返回值
                  log.info("Result: " + result);
              } catch (Exception e) {
                  log.error("RunApiMethod Error: {}:{}::{}",className,methodName,argumentsString);
                  e.printStackTrace();
              }
          }
      
      
          /**
           * 批量调用需要更新缓存的Api
           * */
          public Boolean runApiMethod(){
              List<ApiScheduleEntity> list = getApiScheduleList();
              for (ApiScheduleEntity api : list) {
                  runApiOne(api);
              }
              return true;
          }
      
          /**
           * 添加单个Api定时任务
           * */
          public void addApiScheduledTask(ApiScheduleEntity apiSchedule){
              TimerTask timerTask = new TimerTask() {
                  @Override
                  public void run() {
                      runApiOne(apiSchedule);
                  }
              };
              ScheduledFuture<?> scheduledFuture =  threadPoolTaskScheduler.schedule(timerTask,new CronTrigger(apiSchedule.getCron()));
              //将该实例存入到缓存map中
              if (null != scheduledFuture){
                  threadPool.execute(()->{
                      String key = apiSchedule.getClassName() + ":" + apiSchedule.getMethodName()+ ":" +apiSchedule.getArguments();
                      scheduledFutureMap.put(key,scheduledFuture);
                  });
              }
          }
      
          /**
           * 批量取消长时间未被主动调用的Api定时更新任务
           * */
          public Boolean cancelApiScheduleTask(){
              List<ApiScheduleEntity> list = getLongTimeUnUseApiList(timeout);
              for (ApiScheduleEntity apiSchedule : list) {
                  threadPool.execute(()->{
                      //去除对应定时认为
                      String key = apiSchedule.getClassName() + ":" +apiSchedule.getMethodName() + ":" + apiSchedule.getArguments();
                      ScheduledFuture<?> scheduledFuture = scheduledFutureMap.get(key);
                      if (null != scheduledFuture){
                          scheduledFuture.cancel(true);
                          scheduledFutureMap.remove(key);
                      }
                      //更新数据库状态
                      apiSchedule.setStatus(0);
                      apiScheduleMapper.updateById(apiSchedule);
                  });
              }
              return true;
          }
      }

 调用示例

 完整代码获取

注:如果在使用中出现不能适配的bug,麻烦方便的话请告知我-> 1073260490@qq.com 

评论 1
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值