yarn3.2源码分析之YarnClient与ResourceManager通信之submitApplication（三）

最新推荐文章于 2022-06-02 07:08:02 发布

zhifeng687

最新推荐文章于 2022-06-02 07:08:02 发布

阅读量381

点赞数

分类专栏： Yarn

本文链接：https://blog.csdn.net/qq_26222859/article/details/90036756

版权

概述

submitApplication会经过一系列Event：

app_accepted类型的RMAppEvent
start类型的RMAppAttemptEvent
appAttemptAdd类型的SchedulerEvent
AttemptAdded类型的RMAppAttemptEvent
storeAppAttempt类型的RMStateStoreEvent
AttemptNewSaved类型的RMAppAttemptEvent
launch类型的AMLaucherEvent

在处理app_accepted类型的RMAppEvent的过程中，创建RMAppAttempt。

在处理start类型的RMAppAttemptEvent的过程中，记录RMAppAttempt id到ApplicationMasterService。

在处理 appAttemptAdd类型的SchedulerEvent的过程中，获取AbstractYarnScheduler中的applications集合，记录其相应appId的currentAppAttempt为此RMAppAttempt。

在处理storeAppAttempt类型的RMStateStoreEvent的过程中，将RMAppAttempt封装成ApplicationAttemptStateData，存储到RMStateStore。

在处理AttemptNewSaved类型的RMAppAttemptEvent的过程中，注册ClientTokenMasterKey。

StartAppAttemptTransition处理APP_ACCEPTED类型的RMAppEvent

中央异步调度器AsyncDispatcher将RMAppEvent分发给ApplicationEventDispatcher处理。

ApplicationEventDispatcher获取RMAppEvent对应的RMAppImpl，委托给RMAppImpl处理RMAppEvent。

RMAppImpl注册处理app_accepted类型的RMAppEvent的Transition。

 .addTransition(RMAppState.SUBMITTED, RMAppState.ACCEPTED,
        RMAppEventType.APP_ACCEPTED, new StartAppAttemptTransition())

StartAppAttemptTransition创建RMAppAttemptImpl实例，然后创建start类型的RMAppAttemptEvent，并提交给AsyncDispatcher

private static final class StartAppAttemptTransition extends RMAppTransition {
    @Override
    public void transition(RMAppImpl app, RMAppEvent event) {
      app.createAndStartNewAttempt(false);
    };
  }

private void
      createAndStartNewAttempt(boolean transferStateFromPreviousAttempt) {
    createNewAttempt();
    handler.handle(new RMAppStartAttemptEvent(currentAttempt.getAppAttemptId(),
      transferStateFromPreviousAttempt));
  }

 public RMAppStartAttemptEvent(ApplicationAttemptId appAttemptId,
      boolean transferStateFromPreviousAttempt) {
    super(appAttemptId, RMAppAttemptEventType.START);
    this.transferStateFromPreviousAttempt = transferStateFromPreviousAttempt;
  }

ApplicationAttemptEventDispatcher处理start类型的RMAppAttemptEvent

ResourceManager的内部类RMActiveService为AsyncDispatcher注册RMAppAttemptEvent类型的处理器—— ApplicationAttemptEventDispatcher

 @Override
    protected void serviceInit(Configuration configuration) throws Exception {
 
      // Register event handler for RmAppAttemptEvents
      rmDispatcher.register(RMAppAttemptEventType.class,
          new ApplicationAttemptEventDispatcher(rmContext));
}

中央异步调度器AsyncDispatcher将RMAppAttemptEvent分发给 ApplicationAttemptEventDispatcher处理。

ApplicationAttemptEventDispatcher获取RMAppAttemptEvent对应的RMAppAttemptImpl，委托给RMAppAttemptImpl处理RMAppAttemptEvent。

@Private
  public static final class ApplicationAttemptEventDispatcher implements
      EventHandler<RMAppAttemptEvent> {

    private final RMContext rmContext;

    public ApplicationAttemptEventDispatcher(RMContext rmContext) {
      this.rmContext = rmContext;
    }

    @Override
    public void handle(RMAppAttemptEvent event) {
      ApplicationAttemptId appAttemptId = event.getApplicationAttemptId();
      ApplicationId appId = appAttemptId.getApplicationId();
      RMApp rmApp = this.rmContext.getRMApps().get(appId);
      if (rmApp != null) {
        RMAppAttempt rmAppAttempt = rmApp.getRMAppAttempt(appAttemptId);
        if (rmAppAttempt != null) {
          try {
            rmAppAttempt.handle(event);
          } catch (Throwable t) {
            LOG.error("Error in handling event type " + event.getType()
                + " for applicationAttempt " + appAttemptId, t);
          }
        } else if (rmApp.getApplicationSubmissionContext() != null
            && rmApp.getApplicationSubmissionContext()
            .getKeepContainersAcrossApplicationAttempts()
            && event.getType() == RMAppAttemptEventType.CONTAINER_FINISHED) {
          // For work-preserving AM restart, failed attempts are still
          // capturing CONTAINER_FINISHED events and record the finished
          // containers which will be used by current attempt.
          /