java8的stream写法实现去重

java List去重

一、常规去重

通过List和Set互转，来去掉重复元素。

// 遍历后判断赋给另一个list集合，保持原来顺序
 public static void ridRepeat1(List<String> list) {
  System.out.println("list = [" + list + "]");
  List<String> listNew = new ArrayList<String>();
  for (String str : list) {
   if (!listNew.contains(str)) {
    listNew.add(str);
   }
  }
  System.out.println("listNew = [" + listNew + "]");
 }
 // set集合去重，保持原来顺序
 public static void ridRepeat2(List<String> list) {
  System.out.println("list = [" + list + "]");
  List<String> listNew = new ArrayList<String>();
  Set set = new HashSet();
  for (String str : list) {
   if (set.add(str)) {
    listNew.add(str);
   }
  }
  System.out.println("listNew = [" + listNew + "]");
 }
 // Set去重  由于Set的无序性，不会保持原来顺序
 public static void ridRepeat3(List<String> list) {
  System.out.println("list = [" + list + "]");
  Set set = new HashSet();
  List<String> listNew = new ArrayList<String>();
  set.addAll(list);
  listNew.addAll(set);
  System.out.println("listNew = [" + listNew + "]");
 }
 // Set去重（将ridRepeat3方法缩减为一行） 无序
 public static void ridRepeat4(List<String> list) {
  System.out.println("list = [" + list + "]");
  List<String> listNew = new ArrayList<String>(new HashSet(list));
  System.out.println("listNew = [" + listNew + "]");
 }
 // Set去重并保持原先顺序
 public static void ridRepeat5(List<String> list) {
  System.out.println("list = [" + list + "]");
  List<String> listNew2= new ArrayList<String>(new LinkedHashSet<String>(list));
  System.out.println("listNew = [" + listNew + "]");
 }

二、java8的stream写法实现去重

1、distinct去重

distinct()方法默认是按照父类Object的equals与hashCode工作的
缺点：如果List集合元素为对象，去重不会奏效（除非重写equals与hashcode方法）

//利用java8的stream去重
 List uniqueList = list.stream().distinct().collect(Collectors.toList());
 System.out.println(uniqueList.toString());

2、新特性简写方式

缺点：该方式不能保持原列表顺序而是使用了TreeSet按照字典顺序排序后的列表，如果需求不需要按原顺序则可直接使用。

//根据name属性去重
List<User> lt = list.stream().collect(
  collectingAndThen(
    toCollection(() -> new TreeSet<>(Comparator.comparing(User::getName))), ArrayList::new));
System.out.println("去重后的:" + lt);
//根据name与address属性去重
List<User> lt1 = list.stream().collect(
  collectingAndThen(
    toCollection(() -> new TreeSet<>(Comparator.comparing(o -> o.getName() + ";" + o.getAddress()))), ArrayList::new));
System.out.println("去重后的:" + lt);

当需求中明确有排序要求也可以按上面简写方式再次加工处理使用stream流的sorted()相关API写法。

List<User> lt = list.stream().collect(
    collectingAndThen(
      toCollection(() -> new TreeSet<>(Comparator.comparing(User::getName))),v -> v.stream().sorted().collect(Collectors.toList())));

3、通过 filter() 方法

private static <T> Predicate<T> distinctByKey(Function<? super T, ?> keyExtractor) {
 Set<Object> seen = ConcurrentHashMap.newKeySet();
 return t -> seen.add(keyExtractor.apply(t));
}

参考网址: https://www.jb51.net/article/201632.htm.