【集合】批量处理List中的数据

需求

将List集合中的数据分批次插入到数据库

实现方式

• Java8方式

List<Integer> oldList = Arrays.asList(1, 2, 3, 4, 5, 6, 7);
int limit = (oldList.size() + MAX_NUMBER - 1) / MAX_NUMBER;
//方法一：使用流遍历操作
List<List<Integer>> newList = new ArrayList<>();
Stream.iterate(0, n -> n + 1)
        .limit(limit)
        .forEach(i -> {
            newList.add(oldList.stream()
                    .skip(i * MAX_NUMBER)
                    .limit(MAX_NUMBER)
                    .collect(Collectors.toList()));
        });

//方法二：获取分割后的集合
List<List<Integer>> splitList =
       Stream.iterate(0, n -> n + 1)
               .limit(limit)
               .parallel()
               .map(a -> oldList.stream()
                       .skip(a * MAX_NUMBER)
                       .limit(MAX_NUMBER)
                       .parallel()
                       .collect(Collectors.toList()))
               .collect(Collectors.toList());

• google-guava方式

List<Integer> oldList = Arrays.asList(1, 2, 3, 4, 5, 6, 7);
// 每3个为一组
List<List<Integer>> newList = Lists.partition(oldList, 3);

• apache-common-collection包

List<Integer> oldList = Lists.newArrayList(1, 2, 3, 4, 5, 6, 7, 8);
// 每3个为一组
List<List<Integer>> subs = ListUtils.partition(oldList, 3);

• 手动写一个函数将list等分成多个list

public static <T> List<List<T>> averageAssign(List<T> source, int n) {
    List<List<T>> result = new ArrayList<>();
    //(先计算出余数)
    int remainder = source.size() % n;
    //然后是商
    int number = source.size() / n;
    //偏移量
    int offset = 0;
    for (int i = 0; i < n; i++) {
        List<T> value;
        if (remainder > 0) {
            value = source.subList(i * number + offset, (i + 1) * number + offset + 1);
            remainder--;
            offset++;
        } else {
            value = source.subList(i * number + offset, (i + 1) * number + offset);
        }
        result.add(value);
    }
    return result;
}

参考链接

• java8 Stream 大数据量List分批处理切割