Hive出现 return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTa

return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTas
Hadoop job information for Stage-1: number of mappers: 0; number of reducers: 0
2018-06-06 23:34:46,458 Stage-1 map = 0%, reduce = 0%
Ended Job = job_1528299123875_0001 with errors
Error during job, obtaining debugging information…
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
MapReduce Jobs Launched:
Stage-Stage-1: HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec
2018-06-06 23:48:09,482 Stage-1 map = 0%, reduce = 0%
2018-06-06 23:49:09,973 Stage-1 map = 0%, reduce = 0%
2018-06-06 23:50:10,388 Stage-1 map = 0%, reduce = 0%
2018-06-06 23:51:10,733 Stage-1 map = 0%, reduce = 0%
2018-06-06 23:52:11,271 Stage-1 map = 0%, reduce = 0%
2018-06-06 23:53:11,424 Stage-1 map = 0%, reduce = 0%
2018-06-06 23:54:11,589 Stage-1 map = 0%, reduce = 0%
2018-06-06 23:55:11,793 Stage-1 map = 0%, reduce = 0%
一直出现这个问题!!!!
 我们都知道,启用MapReduce Job是会消耗系统开销的。对于这个问题,从Hive0.10.0版本开始,对于简单的不需要聚合的类似SELECT 《col》 from 《table》 LIMIT n语句,不需要起MapReduce job,直接通过Fetch task获取数据,可以通过下面几种方法实现:
 解决方法:
 hive》 set hive.fetch.task.conversion=more;
hive》 SELECT id, money FROM m limit 10;
OK
1 122
1 185
1 231
1 292
1 316
1 329
1 355
1 356
1 362
1 364
Time taken: 0.138 seconds, Fetched: 10 row(s)
上面 set hive.fetch.task.conversion=more;开启了Fetch任务,所以对于上述简单的列查询不在启用MapReduce job!
方法二:
bin/hive –hiveconf hive.fetch.task.conversion=more
方法三:
上面的两种方法都可以开启了Fetch任务,但是都是临时起作用的;如果你想一直启用这个功能,可以在${HIVE_HOME}/conf/hive-site.xml里面加入以下配置:《property》
《name》hive.fetch.task.conversion《/name》
《value》more《/value》
《description》
Some select queries can be converted to single FETCH task
minimizing latency.Currently the query should be single
sourced not having any subquery and should not have
any aggregations or distincts (which incurrs RS),
lateral views and joins.
1. minimal : SELECT STAR, FILTER on partition columns, LIMIT only
2. more : SELECT, FILTER, LIMIT only (+TABLESAMPLE, virtual columns)
《/description》
《/property》

由于新版本的csdn不太熟练,使用《》 代替<>!

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值