hive youhua

Currently, some relevant settings are:

NameNode Heap Size: 4096mb
DataNode maximum Java heap size: 4096mb
Hadoop maximum Java heap size: 4096mb
Java Options for MapReduce tasks: 768mb

set mapred.child.java.opts = -Xmx12288m;
set mapred.child.java.opts = -XX:-UseGCOverheadLimit;

hive.exec.reducers.bytes.per.reducer

1000000000

size per reducer.The default is 1G, i.e if the input size is 10G, it will use 10 reducers.