Hive的排序

order by

全局排序,因此只有一个reducer,设置了多个reducer也不起作用.

-- 设置reducer的个数为2
set mapred.reduce.tasks=2;
-- 查询
select * from user_install_status_limit order by uptime limit 10;

order by 在hive.mapred.mode = strict 模式下,必须指定 limit,否则执行会报错.  

-- 设置严格模式
set hive.mapred.mode = strict;
-- 全表查询
select * from user_install_status_limit order by uptime;
-- 报错信息
FAILED: SemanticException 1:49 Order by-s without limit are disabled for safety reasons. If you know what you are doing, please make sure that hive.strict.checks.large.query is set to false and that hive.mapred.mode is not set to 'strict' to enable them.. Error encountered near token 'uptime'
-- 加上limit就正常执行.
select * from user_install_status_limit order by uptime limit 10;
-- 取消严格模式
set hive.mapred.mode = nonstrict;

sort by

局部排序,在数据进入reducer前完成排序. 
用sort by进行排序,如果设置mapred.reduce.tasks = 1,那作用和order by是一样的.
如果设置mapred.reduce.tasks > 1,则sort by只保证每个reducer的输出有序,不保证全局有序.

set mapred.reduce.tasks=2;
insert overwrite local directory '/home/hadoop/sort_data' select * from user_install_status_limit sort by uptime;

总结

• 使用 ASC[ascdend] 或 DESC[descend] 关键字来设置查询结果是按升序或降序排列,默认按升序 ASC 排列.
• order by 后可跟多个字段,字段之间用英文逗号隔开,多个字段时,优先级按先后顺序而定.
• 若A、B都升序: order by A,B; 若A、B都用降序,必须用两个DESC: order by A DESC,B DESC;
• 若A用升序,B用降序,SQL该这样写: order by A ASC,B DESC; 也可以这样写(默认):order by A, B DESC;