YARN has been configured to allow a container no smaller than 512 MB and
no larger than 4 GB; the compute nodes have 36 GB of RAM available for containers.
With a virtual memory ratio of 2.1 (the default value), each map can have as much as
3225.6 MB and a reducer can have 5376 MB of virtual memory. Thus our compute
node configured for 36 GB of container space can support up to 24 maps or 14 reducers,
or any combination of mappers and reducers allowed by the available resources on the node.
配置container资源和计算节点mapper\reducer数目
最新推荐文章于 2023-02-07 17:12:40 发布