ForkJoin框架的代码敲一遍,Master-Worker的代码敲一遍,并且优化修改最后统计结果的方法!
ForkJoin提供的是一个用于并行执行任务的框架,把一个大任务分割成若干个小任务,最后汇总每个小任务结果后得到大任务结果的框架。
Master-Worker模式是常用的并行计算模式。它的核心思想是系统由两类进程协作工作:Master进程和Worker进程。 Master负责接收和分配任务,Worker负责处理子任务。当各个Worker子进程处理完成后,会将结果返回给Master,由Master做归纳和总结。 其好处是能将一个大任务分解成若干个小任务,并行执行,从而提高系统的吞吐量。
ForkJoin实例:
package com.bfxy.thread.core.juc; import java.util.concurrent.*; public class MyForkJoin extends RecursiveTask<Integer>{ private int start; private int end; private static final int threshold = 2;//阈值 public MyForkJoin(Integer start, Integer end) { this.start = start; this.end = end; } /** * 实现计算的方法 * @return */ @Override protected Integer compute() { boolean cancomputed = (end-start)<= threshold; int sum = 0; if (cancomputed){ for(int i = start;i <= end; i++){ sum += i; } }else{ int middle = (end+start)/2; MyForkJoin leftForkJoin = new MyForkJoin(start,middle); MyForkJoin rightForkJoin = new MyForkJoin(middle+1,end); //执行左右两边的任务 leftForkJoin.fork(); rightForkJoin.fork(); //等待任务执行完毕后,汇总左右两边的结果 int left = leftForkJoin.join(); int right = rightForkJoin.join(); sum = left + right; } return sum; } public static void main(String[] args) throws ExecutionException, InterruptedException { ForkJoinPool forkJoinPool = new ForkJoinPool(); MyForkJoin mfj = new MyForkJoin(1,100); Future<Integer> result = forkJoinPool.submit(mfj); System.out.println("result:"+result.get()); } }
Master-Worker实例:
测试类:
package com.bfxy.thread.core.design.mymasterworker; import java.util.Random; public class MyMain { public static void main(String[] args) throws InterruptedException { //Runtime.getRuntime().availableProcessors() 当前线程的个数 MyMaster myMaster = new MyMaster(new MyWorker(),Runtime.getRuntime().availableProcessors()); Random r = new Random(); for (int i = 0;i<=100;i++){ MyTask myTask = new MyTask(i,i); myMaster.submit(myTask); } Long start = System.currentTimeMillis(); System.out.println("开始执行"); myMaster.excute(); new Thread(new Runnable() { @Override public void run() { try { myMaster.getCountDownLatch().await(); } catch (InterruptedException e) { e.printStackTrace(); } System.out.println("执行结束"); int ret = myMaster.getResult(); Long end = System.currentTimeMillis(); System.out.println("耗费时间为:"+(end-start)+"毫秒"); System.out.println("总结果为:"+ret); } }).start(); } }
Master类:
package com.bfxy.thread.core.design.mymasterworker; import com.bfxy.thread.core.design.masterworker.Worker; import com.bfxy.thread.core.design.myfuture.RealData; import java.util.HashMap; import java.util.Map; import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.ConcurrentLinkedQueue; import java.util.concurrent.CountDownLatch; public class MyMaster { //1 ConcurrrentLinkedQueue 盛放待处理的任务 private ConcurrentLinkedQueue<MyTask> taskqueue = new ConcurrentLinkedQueue<>(); //2 hashMap<String,Object>worker 盛放每个线程 private HashMap<String,Thread> workers = new HashMap<>(); //3 ConcurrentHashMap盛放每个worker返回的结果 private ConcurrentHashMap<String,Object> resultMap = new ConcurrentHashMap<>(); private CountDownLatch countDownLatch = null; public CountDownLatch getCountDownLatch() { return countDownLatch; } //4构造方法里面要对worker进行一个初始化操作 (遗漏) public MyMaster(MyWorker worker,Integer workcount){//初始化几个worker countDownLatch = new CountDownLatch(workcount); worker.setTaskqueue(this.taskqueue); worker.setResultMap(this.resultMap); worker.setCountDownLatch(countDownLatch); for (int i = 0;i<workcount;i++){ this.workers.put(Integer.toString(i),new Thread(worker)); } } //5 把任务加入到队列中 public void submit(MyTask task){ taskqueue.add(task); } //6 执行任务 public void excute() throws InterruptedException { for (Map.Entry<String,Thread> m:workers.entrySet()){ m.getValue().start(); } } //7 汇总结果 //结束的时候调用汇总的方法 public int getResult(){ int ret = 0; for(Map.Entry<String,Object> m:resultMap.entrySet()){ ret += (Integer) m.getValue(); } return ret; } }
Worker类:
package com.bfxy.thread.core.design.mymasterworker; import java.util.Random; import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.ConcurrentLinkedQueue; import java.util.concurrent.CountDownLatch; public class MyWorker implements Runnable{ //master队列的引用 private ConcurrentLinkedQueue<MyTask> taskqueue; //拥有master中处理任务后结果集合的引用 private ConcurrentHashMap<String,Object> resultMap; private CountDownLatch countDownLatch; public void setCountDownLatch(CountDownLatch countDownLatch) { this.countDownLatch = countDownLatch; } public void setTaskqueue(ConcurrentLinkedQueue<MyTask> taskqueue) { this.taskqueue = taskqueue; } public void setResultMap(ConcurrentHashMap<String, Object> resultMap) { this.resultMap = resultMap; } //实现runnable接口,线程处理 @Override public void run() { while(true){ MyTask task = taskqueue.poll(); if (task == null) { this.countDownLatch.countDown(); break; } Object result = getResult(task); this.resultMap.put(""+task.getId(),result); } } private Random r = new Random(); private Object getResult(MyTask task) { try { Thread.sleep(200 * r.nextInt(10)); } catch (InterruptedException e) { e.printStackTrace(); } int ret = task.getCount(); return ret; } }
任务实体类:
package com.bfxy.thread.core.design.mymasterworker; public class MyTask { public Integer id; public Integer count; public MyTask(Integer id, Integer count) { this.id = id; this.count = count; } public Integer getId() { return id; } public void setId(Integer id) { this.id = id; } public Integer getCount() { return count; } public void setCount(Integer count) { this.count = count; } }