HIVE中EXPLAIN的解读方式
1.执行计划可以知道关键字的执行先后顺序(这里与oracle的会有些不同)
首先来执行一个很简单的sql语句。
hive (dwd)> explain select level,gender,count(*) from user_behavior where channel = 'huawei' group by level,gender;
OK
Explain
STAGE DEPENDENCIES:*--表明当前stage所有的依赖顺序*
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
*--从上文可以看出stage-0 依赖于stage-1,会等待stage-1执行完毕,stage-0才会执行。*
STAGE PLANS:
Stage: Stage-1*--表明当前显示的是stage-1的执行计划,属于整体计划的子计划。*
Tez*--当时用的tez引擎,没留意*
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
DagName: atguigu_20190814062239_a68f9d23-6b82-4d39-b9fd-53704c3382e1:3
Vertices:
Map 1
Map Operator Tree:
TableScan*--先扫描表*
alias: user_behavior*--表名*
Statistics: Num rows: 620000 Data size: 580674000 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (channel = 'huawei') (type: boolean)*--而后过滤*
Statistics: Num rows: 310000 Data size: 290337000 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: level (type: tinyint), gender (type: string)*--继而select2个字段*
outputColumnNames: level, gender
Statistics: Num rows: 310000 Data size: 290337000 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
keys: level (type: tinyint), gender (type: string)*--group by做聚合操作*
mode: hash
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 310000 Data size: 290337000 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: tinyint), _col1 (type: string)*--order by默认升序排序,虽然我们sql中没写,但是会默认做*
sort order: ++
Map-reduce partition columns: _col0 (type: tinyint), _col1 (type: string)
Statistics: Num rows: 310000 Data size: 290337000 Basic stats: COMPLETE Column stats: NONE
value expressions: _col2 (type: bigint)
Reducer 2
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
keys: KEY._col0 (type: tinyint), KEY._col1 (type: string)*--group by做聚合操作,刚才的map1也有同样的操作,推测是预聚合。这里才是最终结果。*
mode: mergepartial
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 155000 Data size: 145168500 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false*--压缩方式没有,false*
Statistics: Num rows: 155000 Data size: 145168500 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
*--至此stage-1结束。接下来进入stage-0*
Stage: Stage-0
Fetch Operator
limit: -1