1.报错日志
2019-10-14 13:05:18,815 INFO [main]: ql.Driver (SessionState.java:printInfo(1087)) - Query ID = tomcat_20191014130505_85923f08-f137-47a4-9516-c26144f3bfda
2019-10-14 13:05:18,815 INFO [main]: ql.Driver (SessionState.java:printInfo(1087)) - Total jobs = 3
2019-10-14 13:05:18,836 INFO [main]: ql.Driver (Driver.java:launchTask(2052)) - Starting task [Stage-17:MAPREDLOCAL] in serial mode
2019-10-14 13:05:18,839 INFO [main]: mr.MapredLocalTask (MapredLocalTask.java:executeInChildVM(175)) - Generating plan file file:/tmp/tomcat/9bcc1db2-89c7-4a4b-bccf-512104e09444/hive_2019-10-14_13-05-14_002_5277827082352258207-1/-local-10008/plan.xml
2019-10-14 13:05:19,300 INFO [main]: mr.MapredLocalTask (MapredLocalTask.java:executeInChildVM(322)) - Executing: /usr/local/cloudera-manager/cloudera/parcels/CDH-5.15.0-1.cdh5.15.0.p0.21/lib/hadoop/bin/hadoop jar /usr/local/cloudera-manager/cloudera/parcels/CDH-5.15.0-1.cdh5.15.0.p0.21/jars/hive-exec-1.1.0-cdh5.15.0.jar org.apache.hadoop.hive.ql.exec.mr.ExecDriver -libjars file:///usr/local/cloudera-manager/cloudera/parcels/CDH-5.15.0-1.cdh5.15.0.p0.21/lib/hbase/hbase-common.jar,file:/usr/local/cloudera-manager/cloudera/parcels/CDH-5.15.0-1.cdh5.15.0.p0.21/lib/hive/auxlib/hive-exec-1.1.0-cdh5.15.0-core.jar,file:///usr/local/cloudera-manager/cloudera/parcels/CDH-5.15.0-1.cdh5.15.0.p0.21/lib/hbase/hbase-hadoop-compat.jar,file:///usr/local/cloudera-manager/cloudera/parcels/CDH-5.15.0-1.cdh5.15.0.p0.21/lib/hbase/lib/htrace-core-3.2.0-incubating.jar,file:///usr/local/cloudera-manager/cloudera/parcels/CDH-5.15.0-1.cdh5.15.0.p0.21/lib/hbase/hbase-client.jar,file:///usr/share/java/mysql-connector-java.jar,file:///usr/local/cloudera-manager/cloudera/parcels/CDH-5.15.0-1.cdh5.15.0.p0.21/lib/hbase/hbase-hadoop2-compat.jar,file:///usr/local/cloudera-manager/cloudera/parcels/CDH-5.15.0-1.cdh5.15.0.p0.21/lib/hbase/lib/htrace-core4-4.0.1-incubating.jar,file:///usr/local/cloudera-manager/cloudera/parcels/CDH-5.15.0-1.cdh5.15.0.p0.21/lib/hbase/hbase-server.jar,file:///usr/local/cloudera-manager/cloudera/parcels/CDH-5.15.0-1.cdh5.15.0.p0.21/lib/hive/lib/hive-hbase-handler-1.1.0-cdh5.15.0.jar,file:///usr/local/cloudera-manager/cloudera/parcels/CDH-5.15.0-1.cdh5.15.0.p0.21/lib/hbase/hbase-protocol.jar,file:/usr/local/cloudera-manager/cloudera/parcels/CDH-5.15.0-1.cdh5.15.0.p0.21/lib/hive/auxlib/hive-exec-core.jar,file:///usr/local/cloudera-manager/cloudera/parcels/CDH-5.15.0-1.cdh5.15.0.p0.21/lib/hbase/lib/htrace-core.jar -localtask -plan file:/tmp/tomcat/9bcc1db2-89c7-4a4b-bccf-512104e09444/hive_2019-10-14_13-05-14_002_5277827082352258207-1/-local-10008/plan.xml -jobconffile file:/tmp/tomcat/9bcc1db2-89c7-4a4b-bccf-512104e09444/hive_2019-10-14_13-05-14_002_5277827082352258207-1/-local-10009/jobconf.xml
2019-10-14 13:05:19,362 INFO [main-LocalTask-MAPREDLOCAL-stderr]: mr.MapredLocalTask (LogRedirector.java:run(65)) - Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0
2019-10-14 13:05:40,711 INFO [main-LocalTask-MAPREDLOCAL-stderr]: mr.MapredLocalTask (LogRedirector.java:run(65)) - 2019-10-14 01:05:40 Processing rows: 200000 Hashtable size: 199999 Memory usage: 105744560 percentage: 0.055
2019-10-14 13:05:47,635 INFO [main-LocalTask-MAPREDLOCAL-stderr]: mr.MapredLocalTask (LogRedirector.java:run(65)) - 2019-10-14 01:05:47 Processing rows: 400000 Hashtable size: 399999 Memory usage: 364921464 percentage: 0.191
2019-10-14 13:05:54,620 INFO [main-LocalTask-MAPREDLOCAL-stderr]: mr.MapredLocalTask (LogRedirector.java:run(65)) - 2019-10-14 01:05:54 Processing rows: 600000 Hashtable size: 599999 Memory usage: 269948592 percentage: 0.141
2019-10-14 13:06:04,205 INFO [main-LocalTask-MAPREDLOCAL-stderr]: mr.MapredLocalTask (LogRedirector.java:run(65)) - 2019-10-14 01:06:04 Processing rows: 900000 Hashtable size: 899999 Memory usage: 298878712 percentage: 0.157
2019-10-14 13:06:18,823 INFO [main-LocalTask-MAPREDLOCAL-stderr]: mr.MapredLocalTask (LogRedirector.java:run(65)) - 2019-10-14 01:06:18 Processing rows: 1300000 Hashtable size: 1299999 Memory usage: 462124912 percentage: 0.242
2019-10-14 13:06:27,881 INFO [main-LocalTask-MAPREDLOCAL-stderr]: mr.MapredLocalTask (LogRedirector.java:run(65)) - 2019-10-14 01:06:24 Processing rows: 1500000 Hashtable size: 1499999 Memory usage: 733931168 percentage: 0.384
2019-10-14 13:06:33,541 INFO [main-LocalTask-MAPREDLOCAL-stderr]: mr.MapredLocalTask (LogRedirector.java:run(65)) - 2019-10-14 01:06:33 Processing rows: 1800000 Hashtable size: 1799999 Memory usage: 814752808 percentage: 0.427
2019-10-14 13:06:42,294 INFO [main-LocalTask-MAPREDLOCAL-stderr]: mr.MapredLocalTask (LogRedirector.java:run(65)) - 2019-10-14 01:06:42 Processing rows: 2000000 Hashtable size: 1999999 Memory usage: 743289136 percentage: 0.389
2019-10-14 13:06:45,241 INFO [main-LocalTask-MAPREDLOCAL-stderr]: mr.MapredLocalTask (LogRedirector.java:run(65)) - 2019-10-14 01:06:45 Processing rows: 2100000 Hashtable size: 2099999 Memory usage: 879025096 percentage: 0.46
2019-10-14 13:06:51,695 INFO [main-LocalTask-MAPREDLOCAL-stderr]: mr.MapredLocalTask (LogRedirector.java:run(65)) - 2019-10-14 01:06:51 Processing rows: 2300000 Hashtable size: 2299999 Memory usage: 903924936 percentage: 0.474
2019-10-14 13:07:04,304 INFO [main-LocalTask-MAPREDLOCAL-stderr]: mr.MapredLocalTask (LogRedirector.java:run(65)) - 2019-10-14 01:07:04 Processing rows: 2700000 Hashtable size: 2699999 Memory usage: 1006784552 percentage: 0.527
2019-10-14 13:07:12,424 INFO [main-LocalTask-MAPREDLOCAL-stderr]: mr.MapredLocalTask (LogRedirector.java:run(65)) - 2019-10-14 01:07:12 Processing rows: 3000000 Hashtable size: 2999999 Memory usage: 1047621488 percentage: 0.549
2019-10-14 13:07:18,192 INFO [main-LocalTask-MAPREDLOCAL-stderr]: mr.MapredLocalTask (LogRedirector.java:run(65)) - 2019-10-14 01:07:18 Processing rows: 3200000 Hashtable size: 3199999 Memory usage: 1171750992 percentage: 0.614
2019-10-14 13:07:24,018 INFO [main-LocalTask-MAPREDLOCAL-stderr]: mr.MapredLocalTask (LogRedirector.java:run(65)) - 2019-10-14 01:07:24 Processing rows: 3400000 Hashtable size: 3399999 Memory usage: 1103519880 percentage: 0.578
2019-10-14 13:07:42,015 INFO [main-LocalTask-MAPREDLOCAL-stderr]: mr.MapredLocalTask (LogRedirector.java:run(65)) - 2019-10-14 01:07:42 Processing rows: 3900000 Hashtable size: 3899999 Memory usage: 1401950664 percentage: 0.734
2019-10-14 13:07:50,611 INFO [main-LocalTask-MAPREDLOCAL-stderr]: mr.MapredLocalTask (LogRedirector.java:run(65)) - 2019-10-14 01:07:50 Processing rows: 4200000 Hashtable size: 4199999 Memory usage: 1442872208 percentage: 0.756
2019-10-14 13:07:56,961 INFO [main-LocalTask-MAPREDLOCAL-stderr]: mr.MapredLocalTask (LogRedirector.java:run(65)) - 2019-10-14 01:07:56 Processing rows: 4400000 Hashtable size: 4399999 Memory usage: 1543953064 percentage: 0.809
2019-10-14 13:08:10,145 INFO [main-LocalTask-MAPREDLOCAL-stderr]: mr.MapredLocalTask (LogRedirector.java:run(65)) - 2019-10-14 01:08:10 Processing rows: 4700000 Hashtable size: 4699999 Memory usage: 1584757192 percentage: 0.83
2019-10-14 13:08:25,193 INFO [main-LocalTask-MAPREDLOCAL-stderr]: mr.MapredLocalTask (LogRedirector.java:run(65)) - 2019-10-14 01:08:25 Processing rows: 4900000 Hashtable size: 4899999 Memory usage: 1529279360 percentage: 0.801
2019-10-14 13:08:37,106 INFO [main-LocalTask-MAPREDLOCAL-stderr]: mr.MapredLocalTask (LogRedirector.java:run(65)) - 2019-10-14 01:08:37 Processing rows: 5100000 Hashtable size: 5099999 Memory usage: 1605237048 percentage: 0.841
2019-10-14 13:08:58,762 INFO [main-LocalTask-MAPREDLOCAL-stderr]: mr.MapredLocalTask (LogRedirector.java:run(65)) - 2019-10-14 01:08:58 Processing rows: 5300000 Hashtable size: 5299999 Memory usage: 1661488104 percentage: 0.87
2019-10-14 13:11:53,183 INFO [main-LocalTask-MAPREDLOCAL-stderr]: mr.MapredLocalTask (LogRedirector.java:run(65)) - Hive Runtime Error: Map local work exhausted memory
2019-10-14 13:12:16,237 ERROR [main]: exec.Task (SessionState.java:printError(1110)) - Execution failed with exit status: 3
2019-10-14 13:12:16,237 ERROR [main]: exec.Task (SessionState.java:printError(1110)) - Obtaining error information
2019-10-14 13:12:16,238 ERROR [main]: exec.Task (SessionState.java:printError(1110)) -
Task failed!
Task ID:
Stage-17
Logs:
2019-10-14 13:12:16,238 ERROR [main]: exec.Task (SessionState.java:printError(1110)) - /tmp/tomcat/hive.log
2019-10-14 13:12:16,239 ERROR [main]: mr.MapredLocalTask (MapredLocalTask.java:executeInChildVM(372)) - Execution failed with exit status: 3
2019-10-14 13:12:16,290 ERROR [main]: ql.Driver (SessionState.java:printError(1110)) - FAILED: Execution Error, return code 3 from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
2019-10-14 13:12:16,291 INFO [main]: ql.Driver (Driver.java:execute(1960)) - Completed executing command(queryId=tomcat_20191014130505_85923f08-f137-47a4-9516-c26144f3bfda); Time taken: 417.479 seconds
2.解决方案
上面的报错,有一句是 Hive Runtime Error: Map local work exhausted memory,可以看出是内存耗尽了,做一下优化
--是否自动转换为mapjoin
set hive.auto.convert.join = true;
--小表的最大文件大小,默认为25000000,即25M
set hive.mapjoin.smalltable.filesize = 25000000;
--是否将多个mapjoin合并为一个
set hive.auto.convert.join.noconditionaltask = true;
--多个mapjoin转换为1个时,所有小表的文件大小总和的最大值。
set hive.auto.convert.join.noconditionaltask.size = 10000000;