running beyond physical memory limits

6/07/19 14:10:30 INFO mapreduce.Job: Task Id : attempt_1459504722049_479821_m_000041_2, Status : FAILED
Container [pid=167394,containerID=container_1459504722049_479821_01_000180] is running beyond physical memory limits. Current usage: 1.1 GB of 1 GB physical memory used; 3.3 GB of 5 GB virtual memory used. Killing container.
Dump of the process-tree for container_1459504722049_479821_01_000180 :
|- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
|- 167394 167381 167394 167394 (bash) 0 0 108654592 301 /bin/bash -c /usr/local/webserver/jdk//bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx2048m -Djava.io.tmpdir=/data11/server/hadoop/yarn/local/usercache/ad/appcache/application_1459504722049_479821/container_1459504722049_479821_01_000180/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/data1/logs/hadoop/yarn/applogs/application_1459504722049_479821/container_1459504722049_479821_01_000180 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA org.apache.hadoop.mapred.YarnChild 10.1.33.89 10669 attempt_1459504722049_479821_m_000041_2 180 1>/data1/logs/hadoop/yarn/applogs/application_1459504722049_479821/container_1459504722049_479821_01_000180/stdout 2>/data1/logs/hadoop/yarn/applogs/application_1459504722049_479821/container_1459504722049_479821_01_000180/stderr

加上这两个参数就可以跑了
-D mapreduce.map.memory.mb=4096 \
-D mapreduce.map.java.opts=-Xmx1800M \

参考于:
http://stackoverflow.com/questions/21005643/container-is-running-beyond-memory-limits

http://www.ericlin.me/application-log-shows-container-is-running-beyond-physical-memory-limits-error

  • 0
    点赞
  • 1
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值