原创作品,允许转载,转载时请务必以超链接形式标明文章
原始出处 、作者信息和本声明。否则将追究法律责任。
http://yaoyinjie.blog.51cto.com/3189782/923378
- select deviceid from t_aa_pc_log where pt='2012-07-07-00' limit 1;
- > select deviceid from t_aa_pc_log where pt='2012-07-07-00' limit 1;
- Total MapReduce jobs = 1
- Launching Job 1 out of 1
- Number of reduce tasks is set to 0 since there's no reduce operator
- Starting Job = job_201205162059_1547550, Tracking URL = http://jt.dc.sh-wgq.sdo.com:50030/jobdetails.jsp?jobid=job_201205162059_1547550
- Kill Command = /home/hdfs/hadoop-current/bin/hadoop job -Dmapred.job.tracker=10.133.10.103:50020 -kill job_201205162059_1547550
- 2012-07-07 16:22:42,570 Stage-1 map = 0%, reduce = 0%
- 2012-07-07 16:22:48,628 Stage-1 map = 80%, reduce = 0%
- 2012-07-07 16:22:49,640 Stage-1 map = 100%, reduce = 0%
- 2012-07-07 16:22:50,654 Stage-1 map = 100%, reduce = 100%
- Ended Job = job_201205162059_1547550
- OK
- 0cf49387a23d9cec25da3d76d6988546
- Time taken: 13.499 seconds
- hive>
- > explain select deviceid from t_aa_pc_log where pt='2012-07-07-00' limit 1;
- OK
- STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 is a root stage
- STAGE PLANS:
- Stage: Stage-1
- Map Reduce
- Alias -> Map Operator Tree:
- t_aa_pc_log
- TableScan
- alias: t_aa_pc_log
- Filter Operator
- predicate:
- expr: (pt = '2012-07-07-00')
- type: boolean
- Select Operator
- expressions:
- expr: deviceid
- type: string
- outputColumnNames: _col0
- Limit
- File Output Operator
- compressed: false
- GlobalTableId: 0
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- Stage: Stage-0
- Fetch Operator
- limit: 1
- Time taken: 0.418 seconds
- protected List<Operator<? extends Serializable>> childOperators;
- protected List<Operator<? extends Serializable>> parentOperators;
- protected boolean done; // 初始化值为false
- mapred.mapper.class = org.apache.hadoop.hive.ql.exec.ExecMapper
- hive.input.format org.apache.hadoop.hive.ql.io.CombineHiveInputFormat
- @Override
- public boolean doNext(K key, V value) throws IOException {
- if (ExecMapper.getDone()) {
- return false;
- }
- return recordReader.next(key, value);
- }
- public void map(Object key, Object value, OutputCollector output,
- Reporter reporter) throws IOException {
- if (oc == null) {
- oc = output;
- rp = reporter;
- mo.setOutputCollector(oc);
- mo.setReporter(rp);
- }
- // reset the execContext for each new row
- execContext.resetRow();
- try {
- if (mo.getDone()) {
- done = true;
- } else {
- // Since there is no concept of a group, we don't invoke
- // startGroup/endGroup for a mapper
- mo.process((Writable)value);
- int childrenDone = 0;
- for (int i = 0; i < childOperatorsArray.length; i++) {
- Operator<? extends Serializable> o = childOperatorsArray[i];
- if (o.getDone()) {
- childrenDone++;
- } else {
- o.process(row, childOperatorsTag[i]);
- }
- }
- // if all children are done, this operator is also done
- if (childrenDone == childOperatorsArray.length) {
- setDone(true);
- }
- @Override
- public void processOp(Object row, int tag) throws HiveException {
- if (currCount < limit) {
- forward(row, inputObjInspectors[tag]);
- currCount++;
- } else {
- setDone(true);
- }
- }