2018-10-22 03:45:41 INFO 2018-10-22 03:45:41,651 Stage-2(job_1540003897972_375058) map = 100%, reduce = 99%, Cumulative CPU 22312.05 sec
2018-10-22 03:46:13 INFO 2018-10-22 03:46:13,311 Stage-2(job_1540003897972_375058) map = 100%, reduce = 100%, Cumulative CPU 22352.18 sec
2018-10-22 03:46:16 INFO MapReduce Total cumulative CPU time: 0 days 6 hours 12 minutes 32 seconds 180 msec
2018-10-22 03:46:16 INFO Stage-2 Elapsed : 568540 ms job_1540003897972_375058
2018-10-22 03:46:16 INFO Ended Job = job_1540003897972_375058
2018-10-22 03:46:18 INFO Execution failed with exit status: 137
2018-10-22 03:46:18 INFO Obtaining error information
2018-10-22 03:46:18 INFO
2018-10-22 03:46:18 INFO Task failed!
2018-10-22 03:46:18 INFO Task ID:
2018-10-22 03:46:18 INFO Stage-20
2018-10-22 03:46:18 INFO
2018-10-22 03:46:18 INFO Logs:
2018-10-22 03:46:18 INFO
2018-10-22 03:46:18 INFO /data0/Logs/dd_edw/hive-0.12.0/hive.log
2018-10-22 03:46:18 INFO FAILED: Execution Error, return code 137 from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
2018-10-22 03:46:18 INFO MapReduce Jobs Launched:
2018-10-22 03:46:19 INFO Stage-1: job_1540003897972_373806 SUCCESS HDFS Read: 2.583 GB HDFS Write: 18.235 GB Elapsed : 4m19s318ms
2018-10-22 03:46:19 INFO Map: Total: 206 Success: 206 Killed: 3 Failed: 0 avgMapTime: 1m6s436ms
2018-10-22 03:46:19 INFO Reduce: Total: 1000 Success: 1000 Killed: 4 Failed: 0 avgReduceTime: 7s423ms avgShuffleTime: 10s628ms avgMergeTime: 1s414ms
可能是由于服务器内存不足引起的, 在进行mapJoin时候, 内存不足以放下小表中的数据
解决方案: 不让其进行mapjoin优化操作
-- 关闭自动装载(尤其是对于内存比较小的机器)
set hive.auto.convert.join= false;