Yarn源码分析之事件异步分发器AsyncDispatcher

事件异步分发器AsyncDispatcher

处理流程

• service初始化时new一个dispatcher用于事件分发
• dispatcher内部有一个queue用于接收和异步处理事件
• dispatcher register各种事件类型及事件处理类，事件处理类中有handle方法用于处理时间
• *

EventType and Handler

ResourceManager.java	Handle Class
NodesListManagerEventType.class	NodesListManager
SchedulerEventType.class	SchedulerEventDispatcher
RMAppEventType.class	ApplicationEventDispatcher
RMAppAttemptEventType.class	ApplicationAttemptEventDispatcher
RMNodeEventType.class	NodeEventDispatcher
RMAppManagerEventType.class	RMAppManager
AMLauncherEventType.class	ApplicationMasterLauncher
ContainerPreemptEventType.class	RMContainerPreemptEventDispatcher
RMFatalEventType.class	ResourceManager.RMFatalEventDispatcher

RMStateStore.java	Handle Class
RMStateStoreEventType.class	ForwardingEventHandler

ContainerManagerImpl.java	Handle Class
ContainerEventType.class	ContainerEventDispatcher
ApplicationEventType.class	ApplicationEventDispatcher
LocalizationEventType.class	ResourceLocalizationService
AuxServicesEventType.class	AuxServices
ContainersMonitorEventType.class	ContainersMonitor
ContainersLauncherEventType.class	ContainersLauncher
LogHandlerEventType.class	LogHandler

NodeManager.java	Handle Class
ContainerManagerEventType.class	ContainerManagerImpl
NodeManagerEventType.class	NodeManager(this)

ResourceLocalizationService.java	Handle Class
LocalizerEventType.class	ResourceLocalizationService.LocalizerTracker

MRAppMaster.java	Handle Class
JobEventType.class	NoopEventHandler
org.apache.hadoop.mapreduce.jobhistory.EventType.class	EventHandler historyService EventHandler NoopEventHandler
ContainerAllocator.EventType.class	ContainerAllocator
JobEventType.class	JobEventDispatcher
TaskEventType.class	TaskEventDispatcher
TaskAttemptEventType.class	TaskAttemptEventDispatcher
CommitterEventType.class	EventHandler
Speculator.EventType.class	SpeculatorEventDispatcher
ContainerAllocator.EventType.class	ContainerAllocator
ContainerLauncher.EventType.class	ContainerLauncher
JobFinishEvent.Type.class	EventHandler

MRAppMaster.java	Handle Class
RMStateStoreEvent.class	RMStateStore

没有通过异步事件处理器处理的事件

RMContainerImpl.java	Handle Class
RMContainerEventType	RMContainerImpl

相关资料

• [YARN]YARN的AsyncDispatcher原理
• Yarn源码分析之事件异步分发器AsyncDispatcher

StateMachineFactory状态机

概念

状态机由一组状态组成，这些状态大体分为三类，分别为初始状态、中间状态和最终状态。状态机首先由初始状态A开始运行，经过一系列的中间状态后到达最终状态，并在最终状态退出，从而形成一个有向无环图。其状态处理的逻辑是收到一个事件，触发状态A到状态B的转换，而转换操作是由事件对应的hook完成的。

处理流程

• 状态机初始化
  
  • 指定状态机初始状态
  • addTransition，transition包括(STATE preState, STATE postState, EVENTTYPE eventType, xxArcTransition hook)
  • 实例化状态机 stateMachine = stateMachineFactory.make(this);
  • 通过上面的异步事件分发器，通过独立的线程获取事件并调应对应处理类的handle方法进行处理。处理过程为根据当前事件类型(type)，事件(event)，执行对应的hook，并从当前状态(oldState)转换到另一种状态(newState)。

NMClientAsync

• 作用
  负责和NodeManagers通信，并对NM的response做异步更新

• 主要功能：

  • startContainer
  • getContainerStatus
  • stopContainer

• 实现流程
  

• 相关代码说明

  • NMClientImpl ：new的单例，包含 cmProxy 和startedContainers
  • ContainerManagementProtocolProxy cmProxy : 负责管理所有的NM 连接proxy
  • ContainerManagementProtocolProxy proxy = cmProxy.getProxy() : 根据container的地址，创建和管理RPC proxy：ContainerManagementProtocolProxyData
  • proxy.getContainerManagementProtocol().startContainers(allRequests); rpc启动container

• 客户端生命周期管理

NMClientAsync asyncClient = 
      NMClientAsync.createNMClientAsync(new MyCallbackhandler());  // 注册消息CallbackHandler() 用于异步处理消息
  asyncClient.init(conf);
  asyncClient.start();
  asyncClient.startContainer(container, containerLaunchContext);
  [... wait for container being started]
  asyncClient.getContainerStatus(container.getId(), container.getNodeId(),
      container.getContainerToken());
  [... handle the status in the callback instance]
  asyncClient.stopContainer(container.getId(), container.getNodeId(),
      container.getContainerToken());
  [... wait for container being stopped]
  asyncClient.stop();

• AM 启动Container执行堆栈

// AM 执行，主要是通过NMClientAsync.startContainer()来异步启动container

ApplicationMaster::run()
nmClientAsyncImpl::startContainerAsync(container, ctx);  // ctx 包含启动脚本
NMClientAsyncImpl::events.put(new StartContainerEvent(container, containerLaunchContext));
NMClientAsyncImpl::StartContainerTransition::container.nmClientAsync.getClient().startContainer(scEvent.getContainer(), scEvent.getContainerLaunchContext());
NMClientImpl::startContainer()
NMClientImpl::proxy = new ContainerManagementProtocolProxyData(rpc, containerManagerBindAddr,containerId, nmTokenCache.getToken(containerManagerBindAddr));
NMClientImpl::StartContainersResponse response = proxy.getContainerManagementProtocol().startContainers(allRequests);
...

// NodeManager 接收RPC请求，并启动container

ContainerManagementImpl::startContainers(StartContainersRequest requests)
ContainerManagementImpl::startContainerInternal(nmTokenIdentifier, containerTokenIdentifier,request);
ContainerManagementImpl::dispatcher.getEventHandler().handle(new ApplicationContainerInitEvent(container));
ApplicationImpl::InitContainerTransition::app.dispatcher.getEventHandler().handle(new ContainerInitEvent(container.getContainerId()));
...
ContainerImpl::handle(ContainerEvent event)
ContainerImpl::sendLaunchEvent();
ContainerImpl::dispatcher.getEventHandler().handle(new ContainersLauncherEvent(this, launcherEvent));
ContainerLauncher::handle(ContainersLauncherEvent event)
ContainerLauncher::ContainerLaunch launch = new ContainerLaunch(context, getConfig(), dispatcher, exec, app, event.getContainer(), dirsHandler, containerManager);
ContainerLaunch::call()
DefaultContainerExecutor::launchContainer()
Shell::execute() -> run() -> runCommand() // 启动子进程执行任务

相关资料

• **YARN源
  码分析之StateMachineFactory状态机 **
• YARN/MRv2 Node Manager深入剖析—Container状态机分析