Hive 中子查询VS条件过滤.

最新推荐文章于 2024-07-31 11:37:15 发布

腹黑客

最新推荐文章于 2024-07-31 11:37:15 发布

阅读量536

点赞数

文章标签： hive big data

本文链接：https://blog.csdn.net/qq_29342297/article/details/120576117

版权

在Hive 2.1.x中，通过对比直接关联和使用子查询进行预过滤的执行计划，发现两者执行效率接近，这得益于Hive引擎的谓词下推优化。

摘要由CSDN通过智能技术生成

基于Hive 2.1.x

首先直接做关联，查看执行计划

-- 先连接后过滤
explain 
select
	count(1)
from student t1 inner join student t2
	on t1.id = t2.id and t2.name = 'shanghai'
where
	t1.city = 'beijing';

1	STAGE DEPENDENCIES:
2	  Stage-5 is a root stage
3	  Stage-2 depends on stages: Stage-5
4	  Stage-0 depends on stages: Stage-2
5	
6	STAGE PLANS:
7	  Stage: Stage-5
8	    Map Reduce Local Work
9	      Alias -> Map Local Tables:
10	        t1 
11	          Fetch Operator
12	            limit: -1
13	      Alias -> Map Local Operator Tree:
14	        t1 
15	          TableScan
16	            alias: t1
17	            filterExpr: (id is not null and (city = 'beijing')) (type: boolean)
18	            Statistics: Num rows: 3 Data size: 9 Basic stats: COMPLETE Column stats: NONE
19	            Filter Operator
20