hive collect_set 结果顺序不一致

HQL 例子

select
  distinct channel
from
(
  select 
    user_id,
    collect_set(channel) as channel
  from test_table 
  group by user_id
) t1
;

结果出现了
[“APP”,“QQ浏览器”]
[“QQ浏览器”,“APP”]

查看代码得知 collect_set 底层用的是 LinkedHashSet 类型

Java 例子

import java.util.LinkedHashSet;

public class Test{
    public static void main(String[] args){
        LinkedHashSet a = new LinkedHashSet();
        a.add("APP");
        a.add("QQ浏览器");
        System.out.println(a);
        LinkedHashSet b = new LinkedHashSet();
        b.add("QQ浏览器");
        b.add("APP");
        System.out.println(b);
    }
}

结果为:
[APP, QQ浏览器]
[QQ浏览器, APP]

如果将LinkedHashSet 改成 HashSet 那么顺序就会一致

解决办法

select
  user_id,
  collect_set(channel) as channel
from
(
  select 
    user_id,
    channel
  from tmp.test_table 
  distribute by user_id
  sort by channel
) t
group by user_id

使用 distribute by + sort by 进行分组排序