Container被kill beyond the ‘VIRTUAL‘ memory limit. Current usage: 308.2 MB of 1 GB physical memory us

报错:

2023-11-29 22:54:36,745 INFO mapreduce.Job: Task Id : attempt_1701267209584_0004_m_000001_2, Status : FAILED
[2023-11-29 22:54:34.910]Container [pid=3211,containerID=container_1701267209584_0004_01_000032] is running 307337728B beyond the 'VIRTUAL' memory limit. Current usage: 308.2 MB of 1 GB physical memory used; 2.4 GB of 2.1 GB virtual memory used. Killing container.
Dump of the process-tree for container_1701267209584_0004_01_000032 :
	|- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
	|- 3211 3210 3211 3211 (bash) 0 0 9789440 288 /bin/bash -c /opt/module/jdk1.8.0_212/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN   -Xmx820m -Djava.io.tmpdir=/opt/module/hadoop-3.1.3/data/nm-local-dir/usercache/qianyihui/appcache/application_1701267209584_0004/container_1701267209584_0004_01_000032/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/opt/module/hadoop-3.1.3/logs/userlogs/application_1701267209584_0004/container_1701267209584_0004_01_000032 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog org.apache.hadoop.mapred.YarnChild 192.168.10.104 43206 attempt_1701267209584_0004_m_000001_2 32 1>/opt/module/hadoop-3.1.3/logs/userlogs/application_1701267209584_0004/container_1701267209584_0004_01_000032/stdout 2>/opt/module/hadoop-3.1.3/logs/userlogs/application_1701267209584_0004/container_1701267209584_0004_01_000032/stderr  
	|- 3222 3211 3211 3211 (java) 244 21 2552406016 78621 /opt/module/jdk1.8.0_212/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx820m -Djava.io.tmpdir=/opt/module/hadoop-3.1.3/data/nm-local-dir/usercache/qianyihui/appcache/application_1701267209584_0004/container_1701267209584_0004_01_000032/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/opt/module/hadoop-3.1.3/logs/userlogs/application_1701267209584_0004/container_1701267209584_0004_01_000032 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog org.apache.hadoop.mapred.YarnChild 192.168.10.104 43206 attempt_1701267209584_0004_m_000001_2 32 

修改yarn配置:

    <property>
        <name>yarn.nodemanager.pmem-check-enabled</name>
        <value>false</value>
    </property>
    <property>
        <name>yarn.nodemanager.vmem-check-enabled</name>
        <value>false</value>
    </property>

解决Yarn在执行程序遇到超出虚拟内存限制,Container被kill

成功运行:

  • 8
    点赞
  • 9
    收藏
    觉得还不错? 一键收藏
  • 1
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 1
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值