- 资源调度框架
(Yet Another Resource Negotiator) - 通用的资源管理系统
- 为上层应用提供统一的资源管理和调度
The fundamental idea of YARN is to split up the functionalities of resource management and job scheduling/monitoring into separate daemons. The idea is to have a global ResourceManager (RM) and per-application ApplicationMaster (AM). An application is either a single job or a DAG of jobs.
YARN的基本思想是将资源管理和作业调度/监视的功能分解为单独的守护进程。我们的想法是拥有一个全局ResourceManager(RM)和每个应用程序ApplicationMaster(AM)。应用程序可以是单个作业,也可以是作业的DAG。
ResourceManager(RM):全局资源管理
ApplicationMaster(AM):应用程序
如果开启spark,就会有spark AM
DAG:相互依赖
The ResourceManager and the NodeManager form the data-computation framework. The ResourceManager is the ultimate authority that arbitrates resources among all the applications in the system. The NodeManager is the per-machine framework agent who is responsible for containers, monitoring their resource usage (cpu, memory, disk, network) and reporting the same to the ResourceManager/Scheduler.
ResourceManager和NodeManager构成了数据计算框架。ResourceManager是在系统中的所有应用程序之间仲裁资源的最终权限NodeManager是每台机器框架代理,负责容器,监视其资源使用情况(CPU,内存,磁盘,网络)并将其报告给ResourceManager / Scheduler。
slave:NodeManager(NM):数据计算框架
The per-application ApplicationMaster is, in effect, a framework specific library and is tasked with negotiating resources from the ResourceManager and working with the NodeManager(s) to execute and monitor the tasks.
每个应用程序ApplicationMaster实际上是一个特定于框架的库,其任务是协调来自ResourceManager的资源,并与NodeManager一起执行和监视任务。