两大表Join时报Map operator initialization failed错误解决方法

1,错误信息 :Map operator initialization failed

TaskAttempt 3 failed, info=[Error: Error while running task ( failure ) : attempt_1596414152423_0018_3_02_000002_3:
java.lang.RuntimeException: java.lang.RuntimeException:
 Map operator initialization failed
	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:381)
	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:75)
	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:62)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:62)
	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:38)
	at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)

2,处理方法在hive中设置即可

set hive.auto.convert.join = false

3,报错原因:mapJoin是先将一张表存放在内存中进行缓存,当如果表的数据过大的时候,内存吃不消进而报错,mapJoin一把处理一张小表和一张大表,新版的hive会自动优化把小表存入内存中进行缓存.如果业务需求必须要用两张大表进行Join,那需要先把mapJoin临时关闭,之后再开启