流处理
自己理解:类似生产线,把需要处理的数据集合看作是生产线上的产品,对流的操作类似经过一道工序,处理后是一个新的流,出来的产品也和之前的不一样,最后由最终操作得到最终的结果,比如装到一个集合里。
菜鸟教程:元素流在管道中经过中间操作(intermediate operation)的处理,最后由最终操作(terminal operation)得到前面处理的结果。
例子:Linux grep,mysql 临时表
集合处理
filter:用于通过设置的条件过滤出元素,返回布尔类型
map:用于映射每个元素到对应的结果,返回一个泛型
collect:将流转换成集合和聚合元素,配合Collectors.静态方法
groupingby:分组,返回Map
limit:个数
parallelStream:并行流,多线程
reduce(累加 ,累乘)
iterate:无限流,
函数式接口
Sync::enable
测试
package top.popchina.lambda;
import java.util.List;
import java.util.stream.Collectors;
import java.util.stream.LongStream;
import java.util.stream.Stream;
public class parallelDemo {
public static void main(String[] args) {
long t1 = 0;
long t2 = 0;
List<Long> collect = Stream.iterate(1L, i -> i + 1).limit(10).collect(Collectors.toList());
System.out.println(collect); //[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
long result1 = Stream.iterate(1L, i -> i + 1).limit(10).reduce(0L, Long::sum);
System.out.println(result1); //55
///
long n = 1000000000L;
t1 = System.currentTimeMillis();
long result2 = Stream.iterate(1L, i -> i + 1).limit(n).reduce(0L, Long::sum);
t2 = System.currentTimeMillis();
System.out.println("result2 = " + result2 + " time = " + (t2 - t1));
t1 = System.currentTimeMillis();
long result3 = Stream.iterate(1L, i -> i + 1).parallel().limit(n).reduce(0L, Long::sum);
t2 = System.currentTimeMillis();
System.out.println("result3 = " + result3 + " time = " + (t2 - t1));
t1 = System.currentTimeMillis();
long result4 = LongStream.rangeClosed(1, n).limit(n).reduce(0L, Long::sum);
t2 = System.currentTimeMillis();
System.out.println("result4 = " + result4 + " time = " + (t2 - t1));
t1 = System.currentTimeMillis();
long result5 = LongStream.rangeClosed(1, n).parallel().limit(n).reduce(0L, Long::sum);
t2 = System.currentTimeMillis();
System.out.println("result5 = " + result5 + " time = " + (t2 - t1));
/*
输出结果
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
55
result2 = 500000500000 time = 41
result3 = 500000500000 time = 176
result4 = 500000500000 time = 22
result5 = 500000500000 time = 12
参与计算的值如果依赖上一步提供的,单线程快
像LongStream等的rangeClosed,可以进行独立运算,多线程在数据量大的情况下比单线程好(多线程的开销要考虑)
不能说指数级提高,只能说提高几倍
*/
System.out.println(" sideEffectSum = " + sideEffectSum(n));
System.out.println(" sideEffectParallelSum = " + sideEffectParallelSum(n));
/*
输出结果
sideEffectSum + 500000500000
sideEffectParallelSum + 345562132828
stream的reduce是线程安全的,性能下降
如果是自己写的函数式接口,如果用到共享变量,不安全,如果没有,安全
*/
}
public static long sideEffectSum(long n) {
Accumulator accumulator = new Accumulator();
long t1 = System.currentTimeMillis();
LongStream.rangeClosed(1, n).forEach(accumulator::add);
long t2 = System.currentTimeMillis();
System.out.print("sideEffectSum time = " + (t2 - t1));
return accumulator.total;
}
public static long sideEffectParallelSum(long n) {
Accumulator accumulator = new Accumulator();
long t1 = System.currentTimeMillis();
LongStream.rangeClosed(1, n).parallel().forEach(accumulator::add);
long t2 = System.currentTimeMillis();
System.out.print("sideEffectParallelSum time = " + (t2 - t1));
return accumulator.total;
}
public static class Accumulator {
public long total = 0;
public void add(long value) {
total += value;
}
}
}