CDH配置

最新推荐文章于 2024-06-17 22:31:37 发布

yangsi_7

最新推荐文章于 2024-06-17 22:31:37 发布

阅读量195

点赞数

分类专栏： CDH 文章标签：大数据

本文链接：https://blog.csdn.net/yangsi_7/article/details/126923025

版权

2 篇文章 0 订阅

订阅专栏

近期Spark作业出现Yarn资源不足执行失败的问题，所以边学习边查看相关的Cloudera Manager配置。特此记录一下重点的关于资源方面的配置事项。

Yarn 相关的性能配置
yarn.nodemanager.resource.cpu-vcores: Container Virtual CPU Cores
yarn.nodemanager.resource.memory-mb: Container Memory
yarn.nodemanager.resource.percentage-physical-cpu-limit: Amount of CPU reserved for all the containers on each node
yarn.scheduler.maximum-allocation-mb:The largest amount of physical memory, in MiB, that can be requested for a container.
yarn.scheduler.maximum-allocation-vcores:The largest number of virtual CPU cores that can be requested for a container. This parameter has no effect prior to CDH 4.4.
yarn.scheduler.fair.assignmultiple
…
ref: https://docs.cloudera.com/documentation/enterprise/6/properties/6.3/topics/cm_props_cdh5160_yarn_mr2included_.html#concept_6.3.x_gateway_props__section_performance_props