由hbase.client.scanner.caching参数引发的血案

 

环境描述

Hadoop 0.20.203.0
Hbase 0.90.3
Hive 0.80.1

 

问题描述

     前几天，在HIVE执行SQL查询的时候出现了一个很奇怪的问题：就是每个SQL（涉及到MapReduce的SQL任务）在执行到某个百分比的时候，整个JOB会出现假死的情况。

 

2012-04-28 18:22:33,661 Stage-1 map = 0%, reduce = 0%
2012-04-28 18:22:59,760 Stage-1 map = 25%, reduce = 0%
2012-04-28 18:23:04,782 Stage-1 map = 38%, reduce = 0%
2012-04-28 18:23:07,796 Stage-1 map = 50%, reduce = 0%
2012-04-28 18:23:08,801 Stage-1 map = 50%, reduce = 8%
2012-04-28 18:23:17,839 Stage-1 map = 50%, reduce = 17%
2012-04-28 18:23:19,848 Stage-1 map = 63%, reduce = 17%
2012-04-28 18:23:32,909 Stage-1 map = 63%, reduce = 21%
2012-04-28 18:23:57,017 Stage-1 map = 75%, reduce = 21%
2012-04-28 18:24:09,075 Stage-1 map = 75%, reduce = 25%
2012-04-28 18:25:09,397 Stage-1 map = 75%, reduce = 25%
2012-04-28 18:26:09,688 Stage-1 map = 75%, reduce = 25%
2012-04-28 18:27:09,980 Stage-1 map = 75%, reduce = 25%
2012-04-28 18:28:10,262 Stage-1 map = 75%, reduce = 25%
2012-04-28 18:29:10,522 Stage-1 map = 75%, reduce = 25%
2012-04-28 18:30:10,742 Stage-1 map = 75%, reduce = 25%
2012-04-28 18:31:10,985 Stage-1 map = 75%, reduce = 25%
2012-04-28 18:32:11,238 Stage-1 map = 75%, reduce = 25%
2012-04-28 18:33:11,467 Stage-1 map = 75%, reduce = 25%
2012-04-28 18:34:11,731 Stage-1 map = 75%, reduce = 25%
2012-04-28 18:35:11,968 Stage-1 map = 75%, reduce = 25%
2012-04-28 18:36:12,213 Stage-1 map = 75%, reduce = 25%
2012-04-28 18:37:12,508 Stage-1 map = 75%, red