Java流控的各种实现方案

Java流控的各种实现方案

1.使用Guava的RateLimiter

原理：RateLimiter使用的是一种叫令牌桶的流控算法，RateLimiter会按照一定的频率往桶里扔令牌，线程拿到令牌才能执行，比如你希望自己的应用程序QPS不要超过1000，那么RateLimiter设置1000的速率后，就会每秒往桶里扔1000个令牌。

代码：

LxRateLimit.java

import java.lang.annotation.*;
import java.util.concurrent.TimeUnit;

@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
@Documented
public @interface LxRateLimit {

    /**
     *
     * @return
     */
    String value() default "";

    /**
     * 每秒向桶中放入令牌的数量   默认最大即不做限流
     * @return
     */
    double perSecond() default Double.MAX_VALUE;

    /**
     * 获取令牌的等待时间  默认0
     * @return
     */
    int timeOut() default 0;

    /**
     * 超时时间单位
     * @return
     */
    TimeUnit timeOutUnit() default TimeUnit.MILLISECONDS;
}

LxRateLimitAspect.java

import com.google.common.util.concurrent.RateLimiter;
import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.Signature;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Pointcut;
import org.aspectj.lang.reflect.MethodSignature;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Component;
import org.springframework.web.bind.annotation.ResponseBody;

import java.lang.reflect.Method;

@Aspect
@Component
public class LxRateLimitAspect {

    private final static Logger logger = LoggerFactory.getLogger(LxRateLimitAspect.class);

    private RateLimiter rateLimiter = RateLimiter.create(Double.MAX_VALUE);

    /**
     * 定义切点
     * 1、通过扫包切入
     * 2、带有指定注解切入
     */
    @Pointcut("@annotation(com.currentlimiting.demo.guava.LxRateLimit)")
    public void checkPointcut() { }

    @ResponseBody
    @Around(value = "checkPointcut()")
    public Object aroundNotice(ProceedingJoinPoint pjp) throws Throwable {
        Signature signature = pjp.getSignature();
        MethodSignature methodSignature = (MethodSignature)signature;
        //获取目标方法
        Method targetMethod = methodSignature.getMethod();
        if (targetMethod.isAnnotationPresent(LxRateLimit.class)) {
            //获取目标方法的@LxRateLimit注解
            LxRateLimit lxRateLimit = targetMethod.getAnnotation(LxRateLimit.class);
            rateLimiter.setRate(lxRateLimit.perSecond());
            if (!rateLimiter.tryAcquire(lxRateLimit.timeOut(), lxRateLimit.timeOutUnit())){
                logger.error("fail");
                return "fail";
            }
        }
        return pjp.proceed();
    }
}

GuavaController.java

import com.currentlimiting.demo.guava.LxRateLimit;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class GuavaController {

    private final static Logger logger = LoggerFactory.getLogger(GuavaController.class);

    @RequestMapping("/guava")
    @LxRateLimit(perSecond = 10)
    public String guava() throws InterruptedException {
        logger.info("guava");
        Thread.sleep(20);
        return "guava";
    }
}

ab测试结果：

条件：QPS=10

[D:\Apache24\bin]$ ab -n 20 -c 20 "http://localhost:8080/guava"

多次模拟并发请求20次，平均有9次失败，11次成功，设置的QPS为10，guava的RateLimiter总会比设置的多1，是因为guava的懒加载的缘故。

工程代码地址：https://github.com/xiedeyantu/currentlimiting/tree/master/guava

2.使用Hystrix限流

介绍：Hystrix是SOA/微服务架构中提供服务隔离、熔断、降级机制的工具/框架。Hystrix是断路器的一种实现，用于高微服务架构的可用性，是防止服务出现雪崩的利器。

原理：Hystrix将请求的逻辑进行封装，一种是把相关逻辑封装在独立的线程中执行实现隔离，另一种是利用信号量来控制并发数实现隔离，如图：

线程池实现：

代码：

HystrixController.java

import com.netflix.hystrix.contrib.javanica.annotation.HystrixCommand;
import com.netflix.hystrix.contrib.javanica.annotation.HystrixProperty;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

import java.util.concurrent.TimeUnit;

@RestController
public class HystrixController {

    private final static Logger logger = LoggerFactory.getLogger(HystrixController.class);
    
    @RequestMapping("/hystrix1")
    @HystrixCommand(
            fallbackMethod = "error",
            threadPoolProperties = { 
                    @HystrixProperty(name = "coreSize", value = "10"),//10个核心线程
                    @HystrixProperty(name = "maxQueueSize", value = "100"),//最大线程数100
                    @HystrixProperty(name = "queueSizeRejectionThreshold", value = "10")},//队列阈值10
            commandProperties = {
                    @HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "10000"), //命令执行超时时间
                    @HystrixProperty(name = "circuitBreaker.requestVolumeThreshold", value = "1000"), //若干10s一个窗口内失败1000次, 则达到触发熔断的最少请求量
                    @HystrixProperty(name = "circuitBreaker.sleepWindowInMilliseconds", value = "1") //断路1s后尝试执行, 默认为5s
            })
    public String hystrix() throws InterruptedException {
        Thread.sleep(500);
        logger.info("hystrix1");
        return "hystrix1";
    }

    public String error() {
        logger.info("fail");
        return "fail";
    }
}

ab测试结果：

条件：线程池核心线程=10，线程池队列=100，队列拒绝策略阈值=10

[D:\Apache24\bin]$ ab -n 42 -c 42 "http://localhost:8080/hystrix"

在多次模拟42个并发请求时，平均失败了20个，理论上核心线程=10（因为最大线程数默认是10），队列拒绝策略阈值=10（小于线程池队列，可以忽略队列值），容量就是20，应该成功22失败20。由于并发测试并不能百分百保证结果完全等于理论值，但是会在理论值上下浮动。hystrix是限制并发数的，所以会比guava的测试结果浮动大一些。

信号量实现：

代码：

HystrixController.java

import com.netflix.hystrix.contrib.javanica.annotation.HystrixCommand;
import com.netflix.hystrix.contrib.javanica.annotation.HystrixProperty;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

import java.util.concurrent.TimeUnit;

@RestController
public class HystrixController {

    @RequestMapping("/hystrix2")
    @HystrixCommand(
            fallbackMethod = "error",

            commandProperties = {
                    @HystrixProperty(name = "execution.isolation.semaphore.maxConcurrentRequests", value = "1"),
                    @HystrixProperty(name = "execution.isolation.strategy", value = "SEMAPHORE")
            })
    public String hystrix2() throws InterruptedException {
        Thread.sleep(500);
        logger.info("hystrix2");
        return "hystrix2";
    }

    public String error() {
        logger.info("fail");
        return "fail";
    }
}

并发测试结果：

条件：最大并发量=1

import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.config.CookieSpecs;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.util.EntityUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.io.IOException;
import java.util.Date;
import java.util.concurrent.CountDownLatch;

public class TestBench {

    private final static Logger logger = LoggerFactory.getLogger(TestBench.class);
    //模拟的并发量
    private static final int CONCURRENT_NUM = 42;
    private static String url = "http://localhost:8080/hystrix2";
    private static CountDownLatch cdl = new CountDownLatch(CONCURRENT_NUM);

    public static void main(String[] args) {
        for (int i = 0; i < CONCURRENT_NUM; i++) {
            new Thread(new Demo()).start();
            cdl.countDown();
        }
    }

    public static class Demo implements Runnable{
        @Override
        public void run() {
            try {
                cdl.await();
            } catch (Exception e) {
                e.printStackTrace();
            }
            //使用工具类发送http请求

            String json2 = requestPost(url, "");
            logger.info(new Date().getTime()+"::"+json2);
        }

    }

    /**
     * 发送POST请求，参数是JSON
     */
    public static String requestPost(String url, String jsonParam){
        logger.info("HttpTool.requestPost 开始 请求url：" + url + ", 参数：" + jsonParam);
        //创建HttpClient对象
        CloseableHttpClient client = HttpClients.createDefault();
        //创建HttpPost对象
        HttpPost httpPost = new HttpPost(url);

        //配置请求参数
        RequestConfig requestConfig = RequestConfig.custom()
                .setCookieSpec(CookieSpecs.DEFAULT)
                .setExpectContinueEnabled(true)
                .setSocketTimeout(5000)
                .setConnectTimeout(5000)
                .setConnectionRequestTimeout(5000)
                .build();

        httpPost.setConfig(requestConfig);

        String respContent = null;

        //设置参数和请求方式
        StringEntity entity = new StringEntity(jsonParam,"UTF-8");//解决中文乱码问题
        entity.setContentEncoding("UTF-8");
        entity.setContentType("application/json");

        httpPost.setEntity(entity);

        HttpResponse resp;
        try {
            //执行请求
            resp = client.execute(httpPost);
            if(resp.getStatusLine().getStatusCode() == 200) {
                HttpEntity responseObj = resp.getEntity();
                respContent = EntityUtils.toString(responseObj,"UTF-8");
            }
        } catch (IOException e) {
            e.printStackTrace();
            logger.info("HttpTool.requestPost 异常 请求url：" + url + ", 参数：" + jsonParam + "，异常信息：" + e);
        }
        logger.info("HttpTool.requestPost 结束 请求url：" + url + ", 参数：" + jsonParam + "");
        //返回数据
        return respContent;
    }
}

在多次模拟42个并发请求时，由于不能严格的模拟测试脚本和server之间的并发请求，通过前后日志输出，来验证，基本符合预期。

hystrix不但可以通过线程池限流，也可以通过信号量来进行限流，不过都是通过控制并发数来实现的，没有直接提供类似QPS的限流方式，不过也可以估算并相互转化。

一般设置标准：

**线程池大小 = 峰值QPS * （99耗时/1s） + 预留线程数 **

工程代码地址：https://github.com/xiedeyantu/currentlimiting/tree/master/hystrix

3.使用resilience限流

介绍：这个库受到Hystrix的启发，但提供了更方便的API和许多其他特性，如RateLimiter(阻塞太频繁的请求)、Bulkhead(避免太多并发请求)等。

原理：Resilience4j的限流器RateLimiter实现了令牌桶限流和基于信号量的固定并发数限流。

代码：

CurrentLimiterConfig.java

import io.github.resilience4j.bulkhead.Bulkhead;
import io.github.resilience4j.bulkhead.BulkheadConfig;
import io.github.resilience4j.ratelimiter.RateLimiter;
import io.github.resilience4j.ratelimiter.RateLimiterConfig;
import org.springframework.context.annotation.Configuration;

import java.time.Duration;

@Configuration
public class CurrentLimiterConfig {

    protected final RateLimiter rateLimiter;
    protected final RateLimiter rateLimiter2;
    protected final Bulkhead bulkhead;

    public CurrentLimiterConfig(){
        rateLimiter = RateLimiter.of("Limiter1",RateLimiterConfig.custom()
                .limitForPeriod(20)
                .limitRefreshPeriod(Duration.ofSeconds(1))
                .timeoutDuration(Duration.ofMillis(5))
                .build());
        rateLimiter2 = RateLimiter.of("Limiter2",RateLimiterConfig.custom()
                .limitForPeriod(100)
                .limitRefreshPeriod(Duration.ofSeconds(5))
                .timeoutDuration(Duration.ofMillis(5))
                .build());
        bulkhead = Bulkhead.of("Bulkhead", BulkheadConfig.custom()
                .maxConcurrentCalls(1)
                .build());
    }
}

RateLimiterController.java

package com.example.resilience;

import io.github.resilience4j.bulkhead.Bulkhead;
import io.github.resilience4j.bulkhead.BulkheadFullException;
import io.github.resilience4j.ratelimiter.RateLimiter;
import io.github.resilience4j.ratelimiter.RequestNotPermitted;
import io.vavr.control.Try;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class RateLimiterController extends CurrentLimiterConfig {

    private final static Logger logger = LoggerFactory.getLogger(RateLimiterController.class);

    private final RateLimiterService service = new RateLimiterService();

    @RequestMapping("/resilience")
    public String resilience() {
        String result = Try.ofSupplier(RateLimiter.decorateSupplier(rateLimiter, service::service))
                .recover(CommonException.class, "fail")
                .recover(RequestNotPermitted.class,"Request not permitted for limiter")
                .get();
        logger.info(result);
        return result;
    }

    @RequestMapping("/resilience2")
    public String resilience2() {
        logger.info("enter resilience2");
        String result = Try.ofSupplier(Bulkhead.decorateSupplier(bulkhead, service::service2))
                .recover(CommonException.class, "fail")
                .recover(RequestNotPermitted.class,"Request not permitted for limiter")
                .recover(BulkheadFullException.class,"Bulkhead name is full")
                .get();
        logger.info(result);
        return result;
    }

}

RateLimiterService.java

package com.example.resilience;

import org.springframework.stereotype.Service;

@Service
public class RateLimiterService {

    public String service() {
        return "resilience";
    }

    public String service2() {
        return "resilience2";
    }
}

ab测试结果：

RateLimiter:

条件：单位时间周期允许个数=20，时间周期=1秒，请求超时时间=5毫秒

[D:\Apache24\bin]$ ab -n 42 -c 42 "http://localhost:8080/resilience"

在多次模拟42个并发请求时，平均失败了20个，符合预期。

并发测试结果：

Bulkhead:

条件：最大并发数=1

在多次模拟42个并发请求时，同hystrix的测试，基本符合预期。

工程代码地址：https://github.com/xiedeyantu/currentlimiting/tree/master/resilience

总结：

如果侧重控制单位时间请求量，可以使用guava的RateLimiter或Resilience的RateLimiter，如果侧重控制并发请求量，可以使用hystrix或Resilience的Bulkhead。

对于一般需求个人认为Resilience的RateLimiter比较好，参数清晰，配置也比较辩解，但是基于Java8的函数式编程需要花时间熟悉一下。