Currently, some relevant settings are:
NameNode Heap Size: 4096mb
DataNode maximum Java heap size: 4096mb
Hadoop maximum Java heap size: 4096mb
Java Options for MapReduce tasks: 768mb
set mapred.child.java.opts = -Xmx12288m;
set mapred.child.java.opts = -XX:-UseGCOverheadLimit;
hive.exec.reducers.bytes.per.reducer | 1000000000 | size per reducer.The default is 1G, i.e if the input size is 10G, it will use 10 reducers. |