java8 stream index,带有批处理的Java 8 Stream

I have a large file that contains a list of items.

I would like to create a batch of items, make an HTTP request with this batch (all of the items are needed as parameters in the HTTP request). I can do it very easily with a for loop, but as Java 8 lover, I want to try writing this with Java 8's Stream framework (and reap the benefits of lazy processing).

Example:

List batch = new ArrayList<>(BATCH_SIZE);

for (int i = 0; i < data.size(); i++) {

batch.add(data.get(i));

if (batch.size() == BATCH_SIZE) process(batch);

}

if (batch.size() > 0) process(batch);

I want to do something a long the line of

lazyFileStream.group(500).map(processBatch).collect(toList())

What would be the best way to do this?

解决方案

You could do it with jOOλ, a library that extends Java 8 streams for single-threaded, sequential stream use-cases:

Seq.seq(lazyFileStream) // Seq

.zipWithIndex() // Seq>

.groupBy(tuple -> tuple.v2 / 500) // Map>

.forEach((index, batch) -> {

process(batch);

});

Behind the scenes, zipWithIndex() is just:

static Seq> zipWithIndex(Stream stream) {

final Iterator it = stream.iterator();

class ZipWithIndex implements Iterator> {

long index;

@Override

public boolean hasNext() {

return it.hasNext();

}

@Override

public Tuple2 next() {

return tuple(it.next(), index++);

}

}

return seq(new ZipWithIndex());

}

... whereas groupBy() is API convenience for:

default Map> groupBy(Function super T, ? extends K> classifier) {

return collect(Collectors.groupingBy(classifier));

}

(Disclaimer: I work for the company behind jOOλ)