hive出现内存溢出_hive错误记录：虚拟内存溢出

Task with the most failures(4):

-----

Task ID:

task_1562543688229_0006_m_000000

URL:

http://0.0.0.0:8088/taskdetails.jsp?jobid=job_1562543688229_0006&tipid=task_1562543688229_0006_m_000000

-----

Diagnostic Messages for this Task:

[2019-07-08 15:32:44.976]Container [pid=100107,containerID=container_1562543688229_0006_01_000005] is running 429783552B beyond the 'VIRTUAL' memory limit. Current usage:

35.8 MB of 1 GB physical memory used; 2.5 GB of 2.1 GB virtual memory used. Killing container.

Dump of the process-tree for container_1562543688229_0006_01_000005 :

|- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE

|- 100117 100107 100107 100107 (java) 33 43 2568744960 8859 /usr/local/java/jdk1.8.0_11/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN

-Xmx820m -Djava.io.tmpdir=/hadoop/pull/nm-local-dir/usercache/root/appcache/application_1562543688229_0006/container_1562543688229_0006_01_000005/tmp -Dlog4j.configuration=

container-log4j.properties -Dyarn.app.container.log.dir=/hadoop/hadoop-3.1.2/logs/userlogs/application_1562543688229_0006/container_1562543688229_0006_01_000005 -Dyarn.app.

container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog org.apache.hadoop.mapred.YarnChild 192.168.176.142 42385 attempt_1562543688229_0006_m_000000_3 5

|- 100107 100105 100107 100107 (bash) 0 0 115896320 308 /bin/bash -c /usr/local/java/jdk1.8.0_11/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN

-Xmx820m -Djava.io.tmpdir=/hadoop/pull/nm-local-dir/usercache/root/appcache/application_1562543688229_0006/container_1562543688229_0006_01_000005/tmp -Dlog4j.configuration=container

-log4j.properties -Dyarn.app.container.log.dir=/hadoop/hadoop-3.1.2/logs/userlogs/application_1562543688229_0006/container_1562543688229_0006_01_000005 -Dyarn.app.container.log.filesize

=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog org.apache.hadoop.mapred.YarnChild 192.168.176.142 42385 attempt_1562543688229_0006_m_000000_3 5 1>

/hadoop/hadoop-3.1.2/logs/userlogs/application_1562543688229_0006/container_1562543688229_0006_01_000005/stdout 2>/hadoop/hadoop-3.1.2/logs/userlogs/application_1562543688229_0006/

container_1562543688229_0006_01_000005/stderr

解决方式 ：

修改hadoop的yarn-site.xml,添加

yarn.nodemanager.vmem-check-enabled

false