CountDownLatch
简介
描述如下：它是一个同步工具类，允许一个或多个线程一直等待，直到其他线程运行完成后再执行。
通过描述，可以清晰的看出，CountDownLatch的两种使用场景：
- 场景1：让多个线程等待
- 场景2：和让单个线程等待。
CountDownLatch的使用和原理解析
详细文章内容：https://zhuanlan.zhihu.com/p/95835099
1、CountDownLatch 概念
CountDownLatch可以使一个获多个线程等待其他线程各自执行完毕后再执行。
CountDownLatch 定义了一个计数器，和一个阻塞队列， 当计数器的值递减为0之前，阻塞队列里面的线程处于挂起状态，当计数器递减到0时会唤醒阻塞队列所有线程，这里的计数器是一个标志，可以表示一个任务一个线程，也可以表示一个倒计时器，CountDownLatch可以解决那些一个或者多个线程在执行之前必须依赖于某些必要的前提业务先执行的场景。
2、CountDownLatch 常用方法说明
CountDownLatch(int count); //构造方法，创建一个值为count 的计数器。
await();//阻塞当前线程，将当前线程加入阻塞队列。
await(long timeout, TimeUnit unit);//在timeout的时间之内阻塞当前线程,时间一过则当前线程可以执行，
countDown();//对计数器进行递减1操作，当计数器递减至0时，当前线程会去唤醒阻塞队列里的所有线程。
案例：同步计数完成任务
①使用awaitTermination
使用线程池的awaitTermination方法同步完成任务
package com.ln.juc.utils.countdownlatch;
import com.ln.juc.utils.RandomUtils;
import com.ln.juc.utils.ThreadUtils;
import com.ln.juc.utils.Tools;
import java.util.Arrays;
import java.util.Random;
import java.util.concurrent.*;
import java.util.stream.IntStream;
import java.util.stream.Stream;
/**
* @ProjectName: java-concurrency
* @Package: com.ln.juc.utils.countdownlatch
* @Name:CountDownLatchExample1
* @Author:linianest
* @CreateTime:2021/1/16 10:59
* @version:1.0
* @Description TODO: 使用线程池的awaitTermination方法同步完成任务
*/
public class CountDownLatchExample1 {
private final static ExecutorService executorService = Executors.newFixedThreadPool(2);
public static void main(String[] args) throws InterruptedException {
int[] data = query();
// System.out.println(Arrays.toString(data));
IntStream.range(0, data.length).forEach(i -> {
executorService.execute(new SimpleRunnable(data, i));
}
);
// RandomUtils.getRandom().
// 关闭线程池
executorService.shutdown();
// 阻塞，等待所有的线程执行完任务
executorService.awaitTermination(1, TimeUnit.HOURS);
System.out.println("all of the work finish done.");
}
static class SimpleRunnable implements Runnable {
private final int[] data;
private final int index;
public SimpleRunnable(int[] data, int index) {
this.data = data;
this.index = index;
}
@Override
public void run() {
ThreadUtils.sleepMILLISECONDS(RandomUtils.getRandom().nextInt(2000));
int value = data[index];
if (value % 2 == 0) {
data[index] = value * 2;
} else {
data[index] = value * 10;
}
System.out.println(Thread.currentThread().getName() + " finished.");
}
}
private static int[] query() {
// return IntStream.rangeClosed(1, 10).toArray();
return new int[]{1, 2, 3, 4, 5, 6, 7, 8, 9, 10};
}
}
②使用CountDownLatch
使用CountDownLatch的计数器的方式统计完成任务
package com.ln.juc.utils.countdownlatch;
import com.ln.juc.utils.RandomUtils;
import com.ln.juc.utils.ThreadUtils;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.stream.IntStream;
/**
* @ProjectName: java-concurrency
* @Package: com.ln.juc.utils.countdownlatch
* @Name:CountDownLatchExample1
* @Author:linianest
* @CreateTime:2021/1/16 10:59
* @version:1.0
* @Description TODO: 使用CountDownLatch的计数器的方式统计完成任务
*/
public class CountDownLatchExample2 {
private final static ExecutorService executorService = Executors.newFixedThreadPool(2);
private final static CountDownLatch latch = new CountDownLatch(10);
public static void main(String[] args) throws InterruptedException {
int[] data = query();
IntStream.range(0, data.length).forEach(i -> {
executorService.execute(new SimpleRunnable(data, i,latch));
}
);
// 等待CountDownLatch的计数器为0
latch.await();
System.out.println("all of the work finish done.");
// 关闭线程池
executorService.shutdown();
}
static class SimpleRunnable implements Runnable {
private final int[] data;
private final int index;
private CountDownLatch latch;
public SimpleRunnable(int[] data, int index, CountDownLatch latch) {
this.data = data;
this.index = index;
this.latch = latch;
}
@Override
public void run() {
ThreadUtils.sleepMILLISECONDS(RandomUtils.getRandom().nextInt(2000));
int value = data[index];
if (value % 2 == 0) {
data[index] = value * 2;
} else {
data[index] = value * 10;
}
System.out.println(Thread.currentThread().getName() + " finished.");
// 统计完成任务，完成一个任务，减少一个计数
latch.countDown();
}
}
private static int[] query() {
// return IntStream.rangeClosed(1, 10).toArray();
return new int[]{1, 2, 3, 4, 5, 6, 7, 8, 9, 10};
}
}
案例：线程间通信
需求
我们在做初始化的任务过程中，需要准备一些初始化后的数据，可以启动新线程执行准备的任务，当完成数据准备工作后，获取得到的数据，才进行初始化后面的任务
代码
package com.ln.juc.utils.countdownlatch;
import com.ln.juc.utils.ThreadUtils;
import java.util.concurrent.CountDownLatch;
/**
* @ProjectName: java-concurrency
* @Package: com.ln.juc.utils.countdownlatch
* @Name:CountDownLatchExample3
* @Author:linianest
* @CreateTime:2021/1/16 13:10
* @version:1.0
* @Description TODO: 串行任务中，并行执行子任务中
*/
/**
* 初始化任务，一个任务需要数据，在执行的过程中，另外一个线程去获取数据
*/
public class CountDownLatchExample3 {
private static final CountDownLatch lath = new CountDownLatch(1);
public static void main(String[] args) throws InterruptedException {
new Thread() {
@Override
public void run() {
System.out.println("Do some initial working.....");
try {
ThreadUtils.sleep(1);
lath.await();
System.out.println("Do other working.....");
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}.start();
new Thread() {
@Override
public void run() {
System.out.println("async prepare for some data.....");
try {
ThreadUtils.sleep(2);
System.out.println("data prepare for done.");
} catch (Exception e) {
e.printStackTrace();
} finally {
lath.countDown();
}
}
}.start();
Thread.currentThread().join();
}
}
执行结果：
Do some initial working.....
async prepare for some data.....
data prepare for done.
Do other working.....
案例：优化报表统计（并行统计）
需求
功能现状
运营系统有统计报表、业务为统计每日的用户新增数量、订单数量、商品的总销量、总销售额…等多项指标统一展示出来，因为数据量比较大，统计指标涉及到的业务范围也比较多，所以这个统计报表的页面一直加载很慢，所以需要对统计报表这块性能需进行优化。
问题分析
统计报表页面涉及到的统计指标数据比较多，每个指标需要单独的去查询统计数据库数据，单个指标只要几秒钟，但是页面的指标有10多个，所以整体下来页面渲染需要将近一分钟。
解决方案
任务时间长是因为统计指标多，而且指标是串行的方式去进行统计的，我们只需要考虑把这些指标从串行化的执行方式改成并行的执行方式，那么整个页面的时间的渲染时间就会大大的缩短， 如何让多个线程同步的执行任务，我们这里考虑使用多线程，每个查询任务单独创建一个线程去执行，这样每个统计指标就可以并行的处理了。
要求
因为主线程需要每个线程的统计结果进行聚合，然后返回给前端渲染，所以这里需要提供一种机制让主线程等所有的子线程都执行完之后再对每个线程统计的指标进行聚合。 这里我们使用CountDownLatch 来完成此功能。
代码
模拟代码
1、分别统计4个指标用户新增数量、订单数量、商品的总销量、总销售额；
2、假设每个指标执行时间为3秒。如果是串行化的统计方式那么总执行时间会为12秒。
3、我们这里使用多线程并行，开启4个子线程分别进行统计
4、主线程等待4个子线程都执行完毕之后，返回结果给前端。
package com.ln.juc.utils.countdownlatch;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.CountDownLatch;
/**
* @ProjectName: java-concurrency
* @Package: com.ln.juc.utils.countdownlatch
* @Name:StatisticalReportExample
* @Author:linianest
* @CreateTime:2021/1/16 12:45
* @version:1.0
* @Description TODO: 统计报表给前端使用
*/
/**
* **模拟代码**
*
* 1、分别统计4个指标用户新增数量、订单数量、商品的总销量、总销售额；
*
* 2、假设每个指标执行时间为3秒。如果是串行化的统计方式那么总执行时间会为12秒。
*
* 3、我们这里使用多线程并行，开启4个子线程分别进行统计
*
* 4、主线程等待4个子线程都执行完毕之后，返回结果给前端。
*/
public class StatisticalReportExample {
//用于聚合所有的统计指标
private static ConcurrentHashMap<String,Integer> map=new ConcurrentHashMap<>();
//创建计数器，这里需要统计4个指标
private static CountDownLatch countDownLatch=new CountDownLatch(4);
public static void main(String[] args) {
//记录开始时间
long startTime=System.currentTimeMillis();
Thread countUserThread=new Thread(new Runnable() {
public void run() {
try {
System.out.println("正在统计新增用户数量");
Thread.sleep(3000);//任务执行需要3秒
map.put("userNumber",1);//保存结果值
countDownLatch.countDown();//标记已经完成一个任务
System.out.println("统计新增用户数量完毕");
} catch (InterruptedException e) {
e.printStackTrace();
}
}
});
Thread countOrderThread=new Thread(new Runnable() {
public void run() {
try {
System.out.println("正在统计订单数量");
Thread.sleep(3000);//任务执行需要3秒
map.put("countOrder",2);//保存结果值
countDownLatch.countDown();//标记已经完成一个任务
System.out.println("统计订单数量完毕");
} catch (InterruptedException e) {
e.printStackTrace();
}
}
});
Thread countGoodsThread=new Thread(new Runnable() {
public void run() {
try {
System.out.println("正在商品销量");
Thread.sleep(3000);//任务执行需要3秒
map.put("countGoods",3);//保存结果值
countDownLatch.countDown();//标记已经完成一个任务
System.out.println("统计商品销量完毕");
} catch (InterruptedException e) {
e.printStackTrace();
}
}
});
Thread countmoneyThread=new Thread(new Runnable() {
public void run() {
try {
System.out.println("正在总销售额");
Thread.sleep(3000);//任务执行需要3秒
map.put("countmoney",4);//保存结果值
countDownLatch.countDown();//标记已经完成一个任务
System.out.println("统计销售额完毕");
} catch (InterruptedException e) {
e.printStackTrace();
}
}
});
//启动子线程执行任务
countUserThread.start();
countGoodsThread.start();
countOrderThread.start();
countmoneyThread.start();
try {
//主线程等待所有统计指标执行完毕
countDownLatch.await();
long endTime=System.currentTimeMillis();//记录结束时间
System.out.println("------统计指标全部完成--------");
System.out.println("统计结果为："+map.toString());
System.out.println("任务总执行时间为"+(endTime-startTime)/1000+"秒");
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
执行结果
正在统计新增用户数量
正在商品销量
正在统计订单数量
正在总销售额
统计销售额完毕
统计订单数量完毕
统计商品销量完毕
------统计指标全部完成--------
统计结果为：{countmoney=4, countOrder=2, userNumber=1, countGoods=3}
任务总执行时间为3秒
统计新增用户数量完毕
Process finished with exit code 0
案例：从DB搬迁到数据湖后，验证数据的完成性
需求：
将海量数据从表中搬迁到数据湖中，假设有100张DB中的表，每张表10亿条数据，搬到到数据湖后，进行数据验证，验证搬迁后的数据完成性
- 验证数据条数
- 验证字段属性是否发生变化
- 表的大小
- 抽样验证数据的指纹（MD5）,数据是否发生变化
要求：完成以上任务，每个阶段日志记录，自动关闭线程池
代码
package com.ln.juc.utils.countdownlatch;
import java.util.ArrayList;
import java.util.List;
import java.util.Random;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.stream.IntStream;
/**
* @ProjectName: java-concurrency
* @Package: com.ln.juc.utils.countdownlatch
* @Name:CountDownLatchExample3
* @Author:linianest
* @CreateTime:2021/1/16 13:10
* @version:1.0
* @Description TODO: 将海量数据从表中搬迁到数据湖中，搬迁完成后进行数据验证，验证搬迁后的数据完成性
*/
public class CountDownLatchExample5 {
private static Random random = new Random(System.currentTimeMillis());
public static void main(String[] args) throws InterruptedException {
Event[] events = {new Event(1), new Event(2)};
// 总体事件
final CountDownLatch latch = new CountDownLatch(events.length);
ExecutorService executorService = Executors.newFixedThreadPool(5);
for (Event event : events) {
List<Table> tables = capture(event);
TaskGroup taskGroup = new TaskGroup(tables.size(), event, latch);
for (Table table : tables) {
TaskBatch taskBatch = new TaskBatch(2, taskGroup);
TrustSourceColumns columnsRunnable = new TrustSourceColumns(table, taskBatch);
TrustSourceRecordCount recordCountRunnable = new TrustSourceRecordCount(table, taskBatch);
executorService.submit(columnsRunnable);
executorService.submit(recordCountRunnable);
}
}
latch.await();
System.out.println("all of the work finish done.");
executorService.shutdown();
}
interface Watcher {
void startWatch();
void done(Table table);
}
/**
* TODO 单个任务表的事件监视器
*/
static class TaskBatch implements Watcher {
private CountDownLatch countDownLatch;
private TaskGroup taskGroup;
public TaskBatch(int size, TaskGroup taskGroup) {
this.countDownLatch = new CountDownLatch(size);
this.taskGroup = taskGroup;
}
@Override
public void startWatch() {
}
@Override
public void done(Table table) {
countDownLatch.countDown();
if (countDownLatch.getCount() == 0) {
System.out.println("The table " + table.tableName + " finished work,[" + table + "]");
taskGroup.done(table);
}
}
}
/**
* TODO 单个事件的监视器
*/
static class TaskGroup implements Watcher {
private CountDownLatch countDownLatch;
private Event event;
private CountDownLatch totalEvent;
public TaskGroup(int size, Event event, CountDownLatch totalEvent) {
this.event = event;
this.countDownLatch = new CountDownLatch(size);
this.totalEvent = totalEvent;
}
@Override
public void startWatch() {
}
@Override
public void done(Table table) {
countDownLatch.countDown();
if (countDownLatch.getCount() == 0) {
// System.out.println("The table " + table.tableName + " finished work,[" + table + "]");
System.out.println("======All of table done in event: " + event.id);
// 完成一个事件任务
totalEvent.countDown();
}
}
}
/**
* 验证数据的记录条数
*/
static class TrustSourceRecordCount implements Runnable {
private final Table table;
private final TaskBatch taskBatch;
public TrustSourceRecordCount(Table table, TaskBatch taskBatch) {
this.table = table;
this.taskBatch = taskBatch;
}
@Override
public void run() {
try {
Thread.sleep(random.nextInt(10_000));
} catch (InterruptedException e) {
e.printStackTrace();
}
table.targetCount = table.sourceRecordCount;
// System.out.println("The table " + table.tableName + " target record count capture done and update the data.");
taskBatch.done(table);
}
}
/**
* 验证数据的字段数据
*/
static class TrustSourceColumns implements Runnable {
private final Table table;
private final TaskBatch taskBatch;
public TrustSourceColumns(Table table, TaskBatch taskBatch) {
this.table = table;
this.taskBatch = taskBatch;
}
@Override
public void run() {
try {
Thread.sleep(random.nextInt(10_000));
} catch (InterruptedException e) {
e.printStackTrace();
}
table.targetColumnSchema = table.sourceColumnSchema;
// System.out.println("The table " + table.tableName + " target columns capture done and update the data.");
taskBatch.done(table);
}
}
static class Event {
int id = 0;
public Event(int id) {
this.id = id;
}
}
/**
*
*/
static class Table {
String tableName;
long sourceRecordCount = 10;
long targetCount;
String sourceColumnSchema = "<table name='A'><column name='coll' type='varchar2'></column></table>";
String targetColumnSchema = "";
public Table(String tableName, long sourceRecordCount) {
this.tableName = tableName;
this.sourceRecordCount = sourceRecordCount;
}
@Override
public String toString() {
return "Table{" +
"tableName='" + tableName + '\'' +
", sourceRecordCount=" + sourceRecordCount +
", targetCount=" + targetCount +
", sourceColumnSchema='" + sourceColumnSchema + '\'' +
", targetColumnSchema='" + targetColumnSchema + '\'' +
'}';
}
}
private static List<Table> capture(Event event) {
List<Table> list = new ArrayList<>();
IntStream.range(0, 10).boxed()
.forEach(i -> {
list.add(new Table("table-" + event.id + "-" + i, i * 1000));
});
return list;
}
}
执行结果要求：完成以上任务，每个阶段日志记录，自动关闭线程池
The table table-1-1 finished work,[Table{tableName='table-1-1', sourceRecordCount=1000, targetCount=1000, sourceColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>', targetColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>'}]
The table table-1-0 finished work,[Table{tableName='table-1-0', sourceRecordCount=0, targetCount=0, sourceColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>', targetColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>'}]
The table table-1-3 finished work,[Table{tableName='table-1-3', sourceRecordCount=3000, targetCount=3000, sourceColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>', targetColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>'}]
The table table-1-2 finished work,[Table{tableName='table-1-2', sourceRecordCount=2000, targetCount=2000, sourceColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>', targetColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>'}]
The table table-1-5 finished work,[Table{tableName='table-1-5', sourceRecordCount=5000, targetCount=5000, sourceColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>', targetColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>'}]
The table table-1-4 finished work,[Table{tableName='table-1-4', sourceRecordCount=4000, targetCount=4000, sourceColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>', targetColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>'}]
The table table-1-6 finished work,[Table{tableName='table-1-6', sourceRecordCount=6000, targetCount=6000, sourceColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>', targetColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>'}]
The table table-1-7 finished work,[Table{tableName='table-1-7', sourceRecordCount=7000, targetCount=7000, sourceColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>', targetColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>'}]
The table table-1-8 finished work,[Table{tableName='table-1-8', sourceRecordCount=8000, targetCount=8000, sourceColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>', targetColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>'}]
The table table-2-0 finished work,[Table{tableName='table-2-0', sourceRecordCount=0, targetCount=0, sourceColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>', targetColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>'}]
The table table-1-9 finished work,[Table{tableName='table-1-9', sourceRecordCount=9000, targetCount=9000, sourceColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>', targetColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>'}]
======All of table done in event: 1
The table table-2-1 finished work,[Table{tableName='table-2-1', sourceRecordCount=1000, targetCount=1000, sourceColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>', targetColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>'}]
The table table-2-3 finished work,[Table{tableName='table-2-3', sourceRecordCount=3000, targetCount=3000, sourceColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>', targetColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>'}]
The table table-2-4 finished work,[Table{tableName='table-2-4', sourceRecordCount=4000, targetCount=4000, sourceColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>', targetColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>'}]
The table table-2-2 finished work,[Table{tableName='table-2-2', sourceRecordCount=2000, targetCount=2000, sourceColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>', targetColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>'}]
The table table-2-6 finished work,[Table{tableName='table-2-6', sourceRecordCount=6000, targetCount=6000, sourceColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>', targetColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>'}]
The table table-2-5 finished work,[Table{tableName='table-2-5', sourceRecordCount=5000, targetCount=5000, sourceColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>', targetColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>'}]
The table table-2-7 finished work,[Table{tableName='table-2-7', sourceRecordCount=7000, targetCount=7000, sourceColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>', targetColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>'}]
The table table-2-8 finished work,[Table{tableName='table-2-8', sourceRecordCount=8000, targetCount=8000, sourceColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>', targetColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>'}]
The table table-2-9 finished work,[Table{tableName='table-2-9', sourceRecordCount=9000, targetCount=9000, sourceColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>', targetColumnSchema='<table name='A'><column name='coll' type='varchar2'></column></table>'}]
======All of table done in event: 2
all of the work finish done.
746
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
