java parallel.invoke,Java 8，在流中使用.parallel会导致OOM错误

In the book Java 8 In Action, section 7.1.1, the authors state that a stream can benefit from parallel processing by adding the function .parallel(). They provide a simple method called parallelSum(int) to illustrate this. I was curious to see how well it worked so I executed this code:

package lambdasinaction.chap7;

import java.util.stream.Stream;

public class ParallelPlay {

public static void main(String[] args) {

System.out.println(parallelSum(100_000_000));

}

public static long parallelSum(long n) {

return Stream.iterate(1L, i -> i + 1)

.limit(n)

.parallel()

.reduce(0L, Long::sum);

}

}

To my surprise, I received this error:

Exception in thread "main" java.lang.OutOfMemoryError

at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)

at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)

at java.lang.reflect.Constructor.newInstance(Unknown Source)

at java.util.concurrent.ForkJoinTask.getThrowableException(Unknown Source)

at java.util.concurrent.ForkJoinTask.reportException(Unknown Source)

at java.util.concurrent.ForkJoinTask.invoke(Unknown Source)

at java.util.stream.SliceOps$1.opEvaluateParallelLazy(Unknown Source)

at java.util.stream.AbstractPipeline.sourceSpliterator(Unknown Source)

at java.util.stream.AbstractPipeline.evaluate(Unknown Source)

at java.util.stream.ReferencePipeline.reduce(Unknown Source)

at lambdasinaction.chap7.ParallelPlay.parallelSum(ParallelPlay.java:15)

at lambdasinaction.chap7.ParallelPlay.main(ParallelPlay.java:8)

Caused by: java.lang.OutOfMemoryError: Java heap space

at java.util.stream.SpinedBuffer.ensureCapacity(Unknown Source)

at java.util.stream.Nodes$SpinedNodeBuilder.begin(Unknown Source)

at java.util.stream.AbstractPipeline.copyInto(Unknown Source)

at java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)

at java.util.stream.SliceOps$SliceTask.doLeaf(Unknown Source)

at java.util.stream.SliceOps$SliceTask.doLeaf(Unknown Source)

at java.util.stream.AbstractShortCircuitTask.compute(Unknown Source)

at java.util.concurrent.CountedCompleter.exec(Unknown Source)

at java.util.concurrent.ForkJoinTask.doExec(Unknown Source)

at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(Unknown Source)

at java.util.concurrent.ForkJoinPool.runWorker(Unknown Source)

at java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source)

I am running Java 1.8.0_45 on Windows 7, SP1 with a four-core processor. What's going on?

解决方案

Here you create an infinite stream and limit it afterwards. There are known problems about processing infinite streams in parallel. In particular there's no way to split the task to equal parts effectively. Internally some heuristics are used which are not well suitable for every task. In your case it's much better to create the finite stream using LongStream.range:

import java.util.stream.LongStream;

public class ParallelPlay {

public static void main(String[] args) {

System.out.println(parallelSum(100_000_000));

}

public static long parallelSum(long n) {

return LongStream.rangeClosed(1, n).parallel().sum();

}

}

In this case Stream engine knows from the very beginning how many elements you have, so it can split the task effectively. Also note that using the LongStream is more effecient as you will have no unnecessary boxing.

In general avoid infinite streams if you can solve your task with finite ones.