okhttp同步请求、异步请求需要注意事项

最近在使用okhttp上遇到了一些小坑，以此记录。

依赖

   <dependency>
            <groupId>com.squareup.okhttp3</groupId>
            <artifactId>okhttp</artifactId>
            <version>4.9.0</version>
   </dependency>

遇到的问题

在做一个简单的爬虫例子的时候，并发去爬取网页内容，首先构造一个OKhttp客户端,我期望的是每次最多有两个线程去请求

  private static final OkHttpClient OK_HTTP_CLIENT;

    static {
        Dispatcher dispatcher = new Dispatcher(new ThreadPoolExecutor(1,
                2,
                2,
                TimeUnit.MINUTES,
                new LinkedBlockingQueue<>(6000), r -> {
            Thread thread = new Thread(r);
            thread.setName("zxc");
            return thread;
        }));
        dispatcher.setMaxRequests(2);
        dispatcher.setMaxRequestsPerHost(2);
        OK_HTTP_CLIENT = new OkHttpClient
                .Builder()
                //设置连接超时时间为30秒
                .connectTimeout(3000, TimeUnit.MILLISECONDS)
                .callTimeout(3000, TimeUnit.MILLISECONDS)
                //自定义线程池任务调度策略
                .dispatcher(dispatcher)
                .connectionPool(new ConnectionPool(2, 1, TimeUnit.MINUTES))
                .build();
    }

然后定义了一个获取网页内容方法

 /**
     * 同步获取网页数据
     *
     * @param url 网站地址
     * @return 网页内容
     * @throws IOException
     */
    public static String fetchHtmlSync(String url) throws IOException {
        try {
            Thread.sleep(20);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        Request request = new Request.Builder().url(url)
                .addHeader("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.51 Safari/537.36")
                .build();
        return OK_HTTP_CLIENT.newCall(request).execute().body().string();
    }

再启动一个客户端，并发去请求我自己搭建的一个本地服务,服务端代码如下，业务逻辑就是休眠一秒后返回

@RestController
@RequestMapping("/demo")
public class DemoController {
    private static volatile  AtomicInteger atomicInteger = new AtomicInteger(0);
    @GetMapping(produces = MediaType.TEXT_PLAIN_VALUE)
    public String get() {
        try {
            Thread.sleep(1000);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        String s = "  sss" + atomicInteger.incrementAndGet();
        System.out.println(s);
        return s;
    }

}

客户端请求代码就是一个简单测试类

public class Test {
    public static void main(String[] args) throws IOException {
        ExecutorService executorService = Executors.newFixedThreadPool(5);
        List<Future<String>> futureList = new ArrayList<>();
        for (int i = 0; i < 10; i++) {
            //同步请求
            Future<String> submit = executorService.submit(() -> MyHttpClient.fetchHtmlSync("http://localhost:8080/demo"));
            //异步请求
            //Future<String> submit = executorService.submit(() -> {
            //    MyCallback myCallback = new MyCallback();
            //    MyHttpClient.fetchHtmlASync("http://localhost:8080/demo", myCallback);
            //    return myCallback.getResult();
            //});
            futureList.add(submit);
        }
        futureList.forEach(f -> {
            String s = null;
            try {
                s = f.get();
            } catch (InterruptedException | ExecutionException e) {
                e.printStackTrace();
            }
            System.out.println(s);
        });
        System.out.println("提交完成");
        executorService.shutdown();

    }

}

发现当我们调用MyHttpClient.fetchHtmlSync方法的时候，每次都会直接返回5个数，观察服务器端也是同时发送了5个请求，因此可以说明，下面这些配置都是对同步请求设置无效的，只针对异步请求有效，因此，同步情况下，我们要控制请求的数量，就要控制我们的ExecutorService executorService = Executors.newFixedThreadPool(5);中指定的线程数量

  Dispatcher dispatcher = new Dispatcher(new ThreadPoolExecutor(1,
                2,
                2,
                TimeUnit.MINUTES,
                new LinkedBlockingQueue<>(6000), r -> {
            Thread thread = new Thread(r);
            thread.setName("zxc");
            return thread;
        }));
        dispatcher.setMaxRequests(2);
        dispatcher.setMaxRequestsPerHost(2);