Yarn：Container is running beyond memory limits

There is a check placed at Yarn level for Virtual and Physical memory usage ratio. Issue is not only that VM doesn't have sufficient physical memory. But it is because Virtual memory usage is more than expected for given physical memory.

问题的原因是：虚拟内存的使用已经超过了物理内存配置比期望的内存。

It can be resolved either by :

1. Disable virtual memory usage check by setting yarn.nodemanager.vmem-check-enabled to false;

2. Increase VM:PM ratio by setting yarn.nodemanager.vmem-pmem-ratio to some higher value.

两种解决办法：

1. 关闭nodemanager虚拟内存检查项配置；

2. 调高虚拟内存和物理内存对应比率；

 

Add following property in yarn-site.xml

<property>
   <name>yarn.nodemanager.vmem-check-enabled</name>
    <value>false</value>
    <description>Whether virtual memory limits will be enforced for containers</description>
  </property>
 <property>
   <name>yarn.nodemanager.vmem-pmem-ratio</name>
    <value>4</value>
    <description>Ratio between virtual memory to physical memory when setting memory limits for containers</description>
  </property>

 

References :

https://issues.apache.org/jira/browse/HADOOP-11364

http://blog.cloudera.com/blog/2014/04/apache-hadoop-yarn-avoiding-6-time-consuming-gotchas/

https://stackoverflow.com/questions/21005643/container-is-running-beyond-memory-limits