多线程并发访问之 Semaphore、CountDownLatch

今天业务需求开发需要开发一个洗数据的小功能，大致业务是有百万级别的数据需要清洗，需要开发一个小功能，循环遍历百万数据调用一个服务接口清晰数据。考虑到接口的并发量，访问量不能太大， 整了一两个小时做了一个并发控制的小程序。

public static class CleanTask implements Runnable {

    private static Semaphore semaphore = new Semaphore(5, true);

    private Runnable runnable;

    private CountDownLatch latch;

    public CleanTask(CountDownLatch latch, Runnable runnable) {
        this.runnable = runnable;
        this.latch = latch;
    }

    @Override
    public void run() {
        try {
            semaphore.acquire();
            runnable.run();
            semaphore.release();
        } catch (InterruptedException ex) {
            LOG.error("线程中断异常", ex);
        } finally {
            latch.countDown();
        }
    }
}

public static void cleanStockData(List<String> stringList) {

    ExecutorService executorService = Executors.newCachedThreadPool();
    CountDownLatch countDownLatch = new CountDownLatch(stringList.size());


    for (String id : stringList) {
        executorService.submit(new CleanTask(countDownLatch, () -> {
            try {
                // 清洗任务
            } catch (Exception ex) {
               // 记录失败case
            }
        }));
    }

    try {
        countDownLatch.await();
    } catch (InterruptedException ex) {
        LOG.error("批跑数据异常", ex);
    }

    // 失败数据记录
}

该小程序能基本实现控制数据清洗速度，防止被请求的服务负载过大异常。