Hadoop中Yarnrunner里面submit Job以及AM生成 至Job处理过程源码解析

参考了一篇文章， 才看懂了Yarnrunner的整个流程： 
http://blog.csdn.net/caodaoxi/article/details/12970993 

网上很多文章写的不是很清楚， 有些到AM的启动的时候就错了， 结合最近俩星期看的Yarnrunner部分的源码 我把我的理解写下来， 打算分三部分 
上： SubmitJob到schduler为ApplicationMaster申请Container 
中：AM到NodeManager启动container 至在NM端执行APPMaster的脚本 
下：启动container去执行Job的过程 

可能中间也会有一些理解的错误， 导致文章并不对， 希望有人能指出来， 毕竟写个博客本身就是为了记录自己对大数据各种工具的学习过程 


先借用上面文章的一部分， 作为一些需要先了解的内容， 不然完全不知道代码在讲什么： 
（1）RMApp：每个application对应一个RMApp对象，保存该application的各种信息。 

（2）RMAppAttempt：每个RMApp可能会对应多个RMAppAttempt对象，这取决于前面的RMAppAttempt是否执行成功，如果不成功，会启动另外一个，直到运行成功。RMAppAttempt对象称为“application执行尝试”，这RMApp与RMAppAttempt关系类似于MapReduce中的task与taskAttempt的关系。 

（3）RMNode：保存各个节点的信息。 

（4）RMContainer：保存各个container的信息。 

2.    事件调度器 

（1）AsyncDispatcher 

中央事件调度器，各个状态机的事件调度器会在中央事件调度器中注册，注册方式信息包括：<事件，事件调度器>。该调度器维护了一个事件队列，它会不断扫描整个队列，取出一个事件，检查事件类型，并交给相应的事件调度器处理。 

（2）各个子事件调度器 
事件类型                       状态机             事件处理器 
RMAppEvent               RMApp ApplicationEventDispatcher 
RMAppAttemptEvent    RMAppAttempt ApplicationAttemptEventDispatcher 
RMNodeEvent               RMNode NodeEventDispatcher 
SchedulerEvent                — SchedulerEventDispatcher 
AMLauncherEvent        — ApplicationMasterLauncher 



好了， 首先我们得从SubmitJob来讲起， 入口是job.waitForCompletion方法， 深入的话会看到最后是调用JobSubmitter类的submitClient.submitJob( 
          jobId, submitJobDir.toString(), job.getCredentials()); 来提交的。 submitClient的申明是ClientProtocol接口， 一共有两个实现类： 
LocalRunner 
YarnRunner 

我之前有一篇文章写得是从LocalRunner这边进入， 然后最后是怎么执行Job的。 
Yarn的话就是就是通过YarnRunner执行的 
我们看一下YarnRunner的submitJob方法： 

主要就是创建了ApplicationSubmissionContext对象， 以及通过ResourceMgrDelegate去提交ApplicationSubmissionContext 

Java代码  

1. public JobStatus submitJob(JobID jobId, String jobSubmitDir, Credentials ts)  
2.   throws IOException, InterruptedException {  
3.       
4.     addHistoryToken(ts);  
5.       
6.     // Construct necessary information to start the MR AM  
7.     //这个appContext很重要， 里面拼接了各种环境变量， 以及启动App Master的脚本  这个对象会一直贯穿于各个类之间， 直到AM启动  
8.     ApplicationSubmissionContext appContext =  
9.       createApplicationSubmissionContext(conf, jobSubmitDir, ts);  
10.   
11.     // Submit to ResourceManager  
12.     try {  
13.       
14.     //通过ResourceMgrDelegate来sumbit这个appContext，  ResourceMgrDelegate类是用来和Resource Manager在通讯的  
15.       ApplicationId applicationId =  
16.           resMgrDelegate.submitApplication(appContext);  
17.   
18.           //这个appMaster并不是我们说的ApplicationMaster对象， 这样的命名刚开始也把我迷惑了。。。  
19.       ApplicationReport appMaster = resMgrDelegate  
20.           .getApplicationReport(applicationId);  
21.       String diagnostics =  
22.           (appMaster == null ?  
23.               "application report is null" : appMaster.getDiagnostics());  
24.       if (appMaster == null  
25.           || appMaster.getYarnApplicationState() == YarnApplicationState.FAILED  
26.           || appMaster.getYarnApplicationState() == YarnApplicationState.KILLED) {  
27.         throw new IOException("Failed to run job : " +  
28.             diagnostics);  
29.       }  
30.       return clientCache.getClient(jobId).getJobStatus(jobId);  
31.     } catch (YarnException e) {  
32.       throw new IOException(e);  
33.     }  
34.   }  

上面讲到了ApplicationSubmissionContext里面存储了启动Application Master的类的脚本， 那么就稍微看一下： 
主要里面都是拼接各种变量和命令的， 所以就略去了大部分， 就留下了最重要的那句: 

Java代码  

1. public ApplicationSubmissionContext createApplicationSubmissionContext(  
2.      Configuration jobConf,  
3.      String jobSubmitDir, Credentials ts) throws IOException {  
4.     
5.   ...  
6.     
7.   //这里才是设定Appmaster类的地方， MRJobConfig.APPLICATION_MASTER_CLASS = org.apache.hadoop.mapreduce.v2.app.MRAppMaster  
8.   //所以最后通过命令在nodemanager那边执行的其实是MRAppMaster类的main方法  
9.   vargs.add(MRJobConfig.APPLICATION_MASTER_CLASS);  
10.    vargs.add("1>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR +  
11.        Path.SEPARATOR + ApplicationConstants.STDOUT);  
12.    vargs.add("2>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR +  
13.        Path.SEPARATOR + ApplicationConstants.STDERR);  
14.     
15.   Vector<String> vargsFinal = new Vector<String>(8);  
16.    // Final command  
17.    StringBuilder mergedCommand = new StringBuilder();  
18.    for (CharSequence str : vargs) {  
19.      mergedCommand.append(str).append(" ");  
20.    }  
21.    vargsFinal.add(mergedCommand.toString());  
22.   
23. ...  
24.   
25. // Setup ContainerLaunchContext for AM container  
26. //根据前面的拼接的命令生成AM的container 在后面会通过这个对象来启动container 从而启动MRAppMaster  
27.    ContainerLaunchContext amContainer =  
28.        ContainerLaunchContext.newInstance(localResources, environment,  
29.          vargsFinal, null, securityTokens, acls);  
30.         
31.   
32.   ...  
33.   //设置AMContainer  
34.   appContext.setAMContainerSpec(amContainer);   
35.     
36.   ...  
37.   
38.    return appContext;  
39.  }  

ApplicationSubmissionContext只需要记住amContainer的启动脚本在里面， 后面会用到。 那么继续看一下ResourceMgrDelegate的submitApplication： 

Java代码  

1. public ApplicationId  
2.     submitApplication(ApplicationSubmissionContext appContext)  
3.         throws YarnException, IOException {  
4.   return client.submitApplication(appContext);  
5. }  

他是通过client来submit的， 这个client实在ResourceMgrDelegate的构造函数里面创建的， 其实就是一个YarnClientImpl对象： 

Java代码  

1. public ResourceMgrDelegate(YarnConfiguration conf) {  
2.   super(ResourceMgrDelegate.class.getName());  
3.   this.conf = conf;  
4.   this.client = YarnClient.createYarnClient();  
5.   init(conf);  
6.   start();  
7. }  

到目前为止， 所有的内容都还是在提交Job的那台Client机器上， 还没有到ResourceManger那边。 

我们看一下YarnClientImpl的submitApplication： 

Java代码  

1. public ApplicationId  
2.      submitApplication(ApplicationSubmissionContext appContext)  
3.          throws YarnException, IOException {  
4.    ApplicationId applicationId = appContext.getApplicationId();  
5.    if (applicationId == null) {  
6.      throw new ApplicationIdNotProvidedException(  
7.          "ApplicationId is not provided in ApplicationSubmissionContext");  
8.    }  
9.   
10. //将appContext设置到一个request里面  
11.    SubmitApplicationRequest request =  
12.        Records.newRecord(SubmitApplicationRequest.class);  
13.    request.setApplicationSubmissionContext(appContext);  
14.   
15.    // Automatically add the timeline DT into the CLC  
16.    // Only when the security and the timeline service are both enabled  
17.    if (isSecurityEnabled() && timelineServiceEnabled) {  
18.      addTimelineDelegationToken(appContext.getAMContainerSpec());  
19.    }  
20.   
21.    //TODO: YARN-1763:Handle RM failovers during the submitApplication call.  
22.   
23.   
24. //通过rmClient提交request， 这个rmClient其实就是ClientRMService类， 是用来和Resource Manager做RPC的call， 通过这个类， 可以直接和RM对话  
25.    rmClient.submitApplication(request);  
26.   
27.    int pollCount = 0;  
28.    long startTime = System.currentTimeMillis();  
29.   
30.   
31. //一直循环， 直到状态变为NEW为止， 如果长时间状态没变， 那么就timeout  
32.    while (true) {  
33.      try {  
34.        YarnApplicationState state =  
35.            getApplicationReport(applicationId).getYarnApplicationState();  
36.        if (!state.equals(YarnApplicationState.NEW) &&  
37.            !state.equals(YarnApplicationState.NEW_SAVING)) {  
38.          LOG.info("Submitted application " + applicationId);  
39.          break;  
40.        }  
41.   
42.        long elapsedMillis = System.currentTimeMillis() - startTime;  
43.        if (enforceAsyncAPITimeout() &&  
44.            elapsedMillis >= asyncApiPollTimeoutMillis) {  
45.          throw new YarnException("Timed out while waiting for application " +  
46.              applicationId + " to be submitted successfully");  
47.        }  
48.   
49.        // Notify the client through the log every 10 poll, in case the client  
50.        // is blocked here too long.  
51.        if (++pollCount % 10 == 0) {  
52.          LOG.info("Application submission is not finished, " +  
53.              "submitted application " + applicationId +  
54.              " is still in " + state);  
55.        }  
56.        try {  
57.          Thread.sleep(submitPollIntervalMillis);  
58.        } catch (InterruptedException ie) {  
59.          LOG.error("Interrupted while waiting for application "  
60.              + applicationId  
61.              + " to be successfully submitted.");  
62.        }  
63.      } catch (ApplicationNotFoundException ex) {  
64.        // FailOver or RM restart happens before RMStateStore saves  
65.        // ApplicationState  
66.        LOG.info("Re-submit application " + applicationId + "with the " +  
67.            "same ApplicationSubmissionContext");  
68.        rmClient.submitApplication(request);  
69.      }  
70.    }  
71.   
72.    return applicationId;  
73.  }  

看一下rmClient也就是ClientRMService的submitApplication： 

Java代码  

1. public SubmitApplicationResponse submitApplication(  
2.      SubmitApplicationRequest request) throws YarnException {  
3.    ApplicationSubmissionContext submissionContext = request  
4.        .getApplicationSubmissionContext();  
5.    ApplicationId applicationId = submissionContext.getApplicationId();  
6.   
7.    // ApplicationSubmissionContext needs to be validated for safety - only  
8.    // those fields that are independent of the RM's configuration will be  
9.    // checked here, those that are dependent on RM configuration are validated  
10.    // in RMAppManager.  
11.   
12.   
13. //开始各种验证 一不开心就不让干活  
14.    String user = null;  
15.    try {  
16.      // Safety  
17.      user = UserGroupInformation.getCurrentUser().getShortUserName();  
18.    } catch (IOException ie) {  
19.      LOG.warn("Unable to get the current user.", ie);  
20.      RMAuditLogger.logFailure(user, AuditConstants.SUBMIT_APP_REQUEST,  
21.          ie.getMessage(), "ClientRMService",  
22.          "Exception in submitting application", applicationId);  
23.      throw RPCUtil.getRemoteException(ie);  
24.    }  
25.   
26. //各种验证  
27.    // Check whether app has already been put into rmContext,  
28.    // If it is, simply return the response  
29.    if (rmContext.getRMApps().get(applicationId) != null) {  
30.      LOG.info("This is an earlier submitted application: " + applicationId);  
31.      return SubmitApplicationResponse.newInstance();  
32.    }  
33.   
34. //继续验证  
35.    if (submissionContext.getQueue() == null) {  
36.      submissionContext.setQueue(YarnConfiguration.DEFAULT_QUEUE_NAME);  
37.    }  
38.    if (submissionContext.getApplicationName() == null) {  
39.      submissionContext.setApplicationName(  
40.          YarnConfiguration.DEFAULT_APPLICATION_NAME);  
41.    }  
42.    if (submissionContext.getApplicationType() == null) {  
43.      submissionContext  
44.        .setApplicationType(YarnConfiguration.DEFAULT_APPLICATION_TYPE);  
45.    } else {  
46.      if (submissionContext.getApplicationType().length() > YarnConfiguration.APPLICATION_TYPE_LENGTH) {  
47.        submissionContext.setApplicationType(submissionContext  
48.          .getApplicationType().substring(0,  
49.            YarnConfiguration.APPLICATION_TYPE_LENGTH));  
50.      }  
51.    }  
52.   
53.    try {  
54.      // call RMAppManager to submit application directly  
55.   //干活  通过rmAppManager提交  
56.      rmAppManager.submitApplication(submissionContext,  
57.          System.currentTimeMillis(), user);  
58.   
59.      LOG.info("Application with id " + applicationId.getId() +   
60.          " submitted by user " + user);  
61.      RMAuditLogger.logSuccess(user, AuditConstants.SUBMIT_APP_REQUEST,  
62.          "ClientRMService", applicationId);  
63.    } catch (YarnException e) {  
64.      LOG.info("Exception in submitting application with id " +  
65.          applicationId.getId(), e);  
66.      RMAuditLogger.logFailure(user, AuditConstants.SUBMIT_APP_REQUEST,  
67.          e.getMessage(), "ClientRMService",  
68.          "Exception in submitting application", applicationId);  
69.      throw e;  
70.    }  
71.   
72.    SubmitApplicationResponse response = recordFactory  
73.        .newRecordInstance(SubmitApplicationResponse.class);  
74.    return response;  
75.  }  

来看一下rmAppManager.submitApplication(submissionContext, 
          System.currentTimeMillis(), user);的具体内容： 

Java代码  

1. protected void submitApplication(  
2.       ApplicationSubmissionContext submissionContext, long submitTime,  
3.       String user) throws YarnException {  
4.     ApplicationId applicationId = submissionContext.getApplicationId();  
5.   
6.     //创建一个RMAppImpl对象 其实就是启动RMApp状态机 以及执行RMAppEvent  
7.     RMAppImpl application =  
8.         createAndPopulateNewRMApp(submissionContext, submitTime, user, false);  
9.     ApplicationId appId = submissionContext.getApplicationId();  
10.   
11.     //如果有安全认证enable的话会走这里， 比如kerberos啥的 我就不这么麻烦了 以看懂为主， 直接到else  
12.     if (UserGroupInformation.isSecurityEnabled()) {  
13.       try {  
14.         this.rmContext.getDelegationTokenRenewer().addApplicationAsync(appId,  
15.             parseCredentials(submissionContext),  
16.             submissionContext.getCancelTokensWhenComplete(),  
17.             application.getUser());  
18.       } catch (Exception e) {  
19.         LOG.warn("Unable to parse credentials.", e);  
20.         // Sending APP_REJECTED is fine, since we assume that the  
21.         // RMApp is in NEW state and thus we haven't yet informed the  
22.         // scheduler about the existence of the application  
23.         assert application.getState() == RMAppState.NEW;  
24.         this.rmContext.getDispatcher().getEventHandler()  
25.           .handle(new RMAppRejectedEvent(applicationId, e.getMessage()));  
26.         throw RPCUtil.getRemoteException(e);  
27.       }  
28.     } else {  
29.       // Dispatcher is not yet started at this time, so these START events  
30.       // enqueued should be guaranteed to be first processed when dispatcher  
31.       // gets started.  
32.         
33.       //启动RMApp的状态机， 这里rmContext其实是resourceManager的Client代理， 这一步就是让去RM端的dispatcher去处理RMAppEventType.START事件  
34.       this.rmContext.getDispatcher().getEventHandler()  
35.         .handle(new RMAppEvent(applicationId, RMAppEventType.START));  
36.     }  
37.   }  

在文章的开头有写“事件调度器”， 在resourcemanager那边会有AsyncDispatcher来调度所有事件， 这里的话会通过ApplicationEventDispatcher去做RmAppImpl的transition方法， 看一下RmAppImpl类的初始化的时候的各种event和transition： 

Java代码  

1. private static final StateMachineFactory<RMAppImpl,  
2.                                            RMAppState,  
3.                                            RMAppEventType,  
4.                                            RMAppEvent> stateMachineFactory  
5.                                = new StateMachineFactory<RMAppImpl,  
6.                                            RMAppState,  
7.                                            RMAppEventType,  
8.                                            RMAppEvent>(RMAppState.NEW)  
9.   
10.   
11.      // Transitions from NEW state  
12.     .addTransition(RMAppState.NEW, RMAppState.NEW,  
13.         RMAppEventType.NODE_UPDATE, new RMAppNodeUpdateTransition())  
14.     .addTransition(RMAppState.NEW, RMAppState.NEW_SAVING,  
15.         RMAppEventType.START, new RMAppNewlySavingTransition())  
16.   
17. ...  
18.   
19.  //实在太长了， 到这里我们可以看到上面已经捕捉了RMAppEventType.START事件， 会把RMApp的状态从NEW变成NEW_SAVING， 调用RMAppNewlySavingTransition方法  

我们看一下RMAppNewlySavingTransition里面做了什么： 

Java代码  

1. private static final class RMAppNewlySavingTransition extends RMAppTransition {  
2.   @Override  
3.   public void transition(RMAppImpl app, RMAppEvent event) {  
4.   
5.     // If recovery is enabled then store the application information in a  
6.     // non-blocking call so make sure that RM has stored the information  
7.     // needed to restart the AM after RM restart without further client  
8.     // communication  
9.     LOG.info("Storing application with id " + app.applicationId);  
10.     app.rmContext.getStateStore().storeNewApplication(app);  
11.   }  
12. }  

只做了storeNewApplication(app)这个动作： 

Java代码  

1. public synchronized void storeNewApplication(RMApp app) {  
2.   ApplicationSubmissionContext context = app  
3.                                           .getApplicationSubmissionContext();  
4.   assert context instanceof ApplicationSubmissionContextPBImpl;  
5.   ApplicationState appState =  
6.       new ApplicationState(app.getSubmitTime(), app.getStartTime(), context,  
7.         app.getUser());  
8.   
9. 触发了RMStateStore的STORE_APP事件  
10.   dispatcher.getEventHandler().handle(new RMStateStoreAppEvent(appState));  
11. }  

RMStateStore的状态机transition定义： 

Java代码  

1.   private static final StateMachineFactory<RMStateStore,  
2.                                            RMStateStoreState,  
3.                                            RMStateStoreEventType,   
4.                                            RMStateStoreEvent>  
5.       stateMachineFactory = new StateMachineFactory<RMStateStore,  
6.                                                     RMStateStoreState,  
7.                                                     RMStateStoreEventType,  
8.                                                     RMStateStoreEvent>(  
9.       RMStateStoreState.DEFAULT)  
10.       .addTransition(RMStateStoreState.DEFAULT, RMStateStoreState.DEFAULT,  
11.           RMStateStoreEventType.STORE_APP, new StoreAppTransition())  
12.   
13. ...  
14.   
15. //看到这里执行StoreAppTransition的方法  

继续看下去StoreAppTransition： 

Java代码  

1. private static class StoreAppTransition  
2.     implements SingleArcTransition<RMStateStore, RMStateStoreEvent> {  
3.   @Override  
4.   public void transition(RMStateStore store, RMStateStoreEvent event) {  
5.     if (!(event instanceof RMStateStoreAppEvent)) {  
6.       // should never happen  
7.       LOG.error("Illegal event type: " + event.getClass());  
8.       return;  
9.     }  
10.     ApplicationState appState = ((RMStateStoreAppEvent) event).getAppState();  
11.     ApplicationId appId = appState.getAppId();  
12.     ApplicationStateData appStateData = ApplicationStateData  
13.         .newInstance(appState);  
14.     LOG.info("Storing info for app: " + appId);  
15.     try {  
16.    
17.  //1. store application state  
18.       store.storeApplicationStateInternal(appId, appStateData);  
19.   
20.  // 2.更改状态至RMAppEventType.APP_NEW_SAVED  
21.       store.notifyApplication(new RMAppEvent(appId,  
22.              RMAppEventType.APP_NEW_SAVED));  
23.     } catch (Exception e) {  
24.       LOG.error("Error storing app: " + appId, e);  
25.       store.notifyStoreOperationFailed(e);  
26.     }  
27.   };  
28. }  
29.   
30. private void notifyApplication(RMAppEvent event) {  
31.   
32. 调用dispatcher触发RMAppEventType.APP_NEW_SAVED  
33.   rmDispatcher.getEventHandler().handle(event);  
34. }  

在RMAppImpl中我们可以看到RMAppEventType.APP_NEW_SAVED会触发AddApplicationToSchedulerTransition方法 

Java代码  

1. .addTransition(RMAppState.NEW_SAVING, RMAppState.SUBMITTED,  
2.     RMAppEventType.APP_NEW_SAVED, new AddApplicationToSchedulerTransition())  

看一下AddApplicationToSchedulerTransition： 

Java代码  

1.  private static final class AddApplicationToSchedulerTransition extends  
2.      RMAppTransition {  
3.    @Override  
4.    public void transition(RMAppImpl app, RMAppEvent event) {  
5.   
6. //这里会去调用scheduler的APP_ADDED scheduler类有好几个， 比如说FifoScheduler或者FairScheduler  
7. //我们就看一下FifoScheduler的APP_ADDED事件吧  
8.      app.handler.handle(new AppAddedSchedulerEvent(app.applicationId,  
9.        app.submissionContext.getQueue(), app.user,  
10.        app.submissionContext.getReservationID()));  
11.    }  
12.  }  

我们到FifoScheduler的APP_ADDED事件看看： 

Java代码  

1. case APP_ADDED:  
2. {  
3.   AppAddedSchedulerEvent appAddedEvent = (AppAddedSchedulerEvent) event;  
4.   addApplication(appAddedEvent.getApplicationId(),  
5.     appAddedEvent.getQueue(), appAddedEvent.getUser(),  
6.     appAddedEvent.getIsAppRecovering());  
7. }  

会去调用scheduler的addApplication方法, 看一下： 

Java代码  

1. public synchronized void addApplication(ApplicationId applicationId,  
2.     String queue, String user, boolean isAppRecovering) {  
3.    
4.  //创建一个SchedulerApplication  
5.   SchedulerApplication<FiCaSchedulerApp> application =  
6.       new SchedulerApplication<FiCaSchedulerApp>(DEFAULT_QUEUE, user);  
7.   
8. /Application放到Scheduler里面  
9.   applications.put(applicationId, application);  
10.   metrics.submitApp(user);  
11.   LOG.info("Accepted application " + applicationId + " from user: " + user  
12.       + ", currently num of applications: " + applications.size());  
13.   if (isAppRecovering) {  
14.     if (LOG.isDebugEnabled()) {  
15.       LOG.debug(applicationId + " is recovering. Skip notifying APP_ACCEPTED");  
16.     }  
17.   } else {  
18.   
19. /通过resourceManager的dispatcher去触发RMAppEventType.APP_ACCEPTED   
20.     rmContext.getDispatcher().getEventHandler()  
21.       .handle(new RMAppEvent(applicationId, RMAppEventType.APP_ACCEPTED));  
22.   }  
23. }  

那么我们就要再回到RMAPPImpl去看看RMAppEventType.APP_ACCEPTED做了什么： 

Java代码  

1. .addTransition(RMAppState.SUBMITTED, RMAppState.ACCEPTED,  
2.         RMAppEventType.APP_ACCEPTED, new StartAppAttemptTransition())  

会去执行StartAppAttemptTransition方法， 这个方法其实就是尝试去启动一次Application， 如果失败 还会尝试， 直到尝试的次数到达最大尝试次数为止 

看一下StartAppAttemptTransition方法： 

Java代码  

1. private static final class StartAppAttemptTransition extends RMAppTransition {  
2.   @Override  
3.   public void transition(RMAppImpl app, RMAppEvent event) {  
4.   
5. /创建启动attempt  
6.     app.createAndStartNewAttempt(false);  
7.   };  
8. }  
9.   
10. //在createAndStartNewAttempt中创建了RMAppAttempt， 然后去触发RMAppAttemptEventType.START事件  
11.   private void  
12.     createAndStartNewAttempt(boolean transferStateFromPreviousAttempt) {  
13.   createNewAttempt();  
14.   handler.handle(new RMAppStartAttemptEvent(currentAttempt.getAppAttemptId(),  
15.     transferStateFromPreviousAttempt));  
16. }  
17.   
18.   
19.   private void createNewAttempt() {  
20.   ApplicationAttemptId appAttemptId =  
21.       ApplicationAttemptId.newInstance(applicationId, attempts.size() + 1);  
22.   RMAppAttempt attempt =  
23.       new RMAppAttemptImpl(appAttemptId, rmContext, scheduler, masterService,  
24.         submissionContext, conf,  
25.         // The newly created attempt maybe last attempt if (number of  
26.         // previously failed attempts(which should not include Preempted,  
27.         // hardware error and NM resync) + 1) equal to the max-attempt  
28.         // limit.  
29.     
30.   //每次尝试 +1 知道最大次数到了为止  
31.         maxAppAttempts == (getNumFailedAppAttempts() + 1), amReq);  
32.   attempts.put(appAttemptId, attempt);  
33.   currentAttempt = attempt;  
34. }  

这里就会去RMAPPAttemptImpl 里面去触发RMAppAttemptEventType.START事件， 看一下这个状态机里面的transition是怎么样的： 

Java代码  

1. private static final StateMachineFactory<RMAppAttemptImpl,  
2.                                          RMAppAttemptState,  
3.                                          RMAppAttemptEventType,  
4.                                          RMAppAttemptEvent>  
5.      stateMachineFactory  = new StateMachineFactory<RMAppAttemptImpl,  
6.                                           RMAppAttemptState,  
7.                                           RMAppAttemptEventType,  
8.                                    RMAppAttemptEvent>(RMAppAttemptState.NEW)  
9.   
10.      // Transitions from NEW State  
11.     .addTransition(RMAppAttemptState.NEW, RMAppAttemptState.SUBMITTED,  
12.         RMAppAttemptEventType.START, new AttemptStartedTransition())  
13.     
14.   //回去调用本类的AttemptStartedTransition方法  

看一下本类的AttemptStartedTransition方法： 

Java代码  

1. private static final class AttemptStartedTransition extends BaseTransition {  
2. Override  
3.   public void transition(RMAppAttemptImpl appAttempt,  
4.       RMAppAttemptEvent event) {  
5.   
6.    boolean transferStateFromPreviousAttempt = false;  
7.     if (event instanceof RMAppStartAttemptEvent) {  
8.       transferStateFromPreviousAttempt =  
9.           ((RMAppStartAttemptEvent) event)  
10.             .getTransferStateFromPreviousAttempt();  
11.     }  
12.     appAttempt.startTime = System.currentTimeMillis();  
13.   
14.     // Register with the ApplicationMasterService  
15.     appAttempt.masterService  
16.         .registerAppAttempt(appAttempt.applicationAttemptId);  
17.   
18.     if (UserGroupInformation.isSecurityEnabled()) {  
19.       appAttempt.clientTokenMasterKey =  
20.           appAttempt.rmContext.getClientToAMTokenSecretManager()  
21.             .createMasterKey(appAttempt.applicationAttemptId);  
22.     }  
23.   
24.     // Add the applicationAttempt to the scheduler and inform the scheduler  
25.     // whether to transfer the state from previous attempt.  
26.    
27.  //将applicationAttempt触发scheduler的APP_ATTEMPT_ADDED事件， 就是FifoScheduler的APP_ATTEMPT_ADDED事件  
28.    
29.     appAttempt.eventHandler.handle(new AppAttemptAddedSchedulerEvent(  
30.       appAttempt.applicationAttemptId, transferStateFromPreviousAttempt));  
31.   }  
32. }  

我们回到FifoScheduler里面看一下APP_ATTEMPT_ADDED做了什么： 

Java代码  

1.    case APP_ATTEMPT_ADDED:  
2.    {  
3.      AppAttemptAddedSchedulerEvent appAttemptAddedEvent =  
4.          (AppAttemptAddedSchedulerEvent) event;  
5.         
6.       //执行addApplicationAttempt方法  
7.      addApplicationAttempt(appAttemptAddedEvent.getApplicationAttemptId(),  
8.        appAttemptAddedEvent.getTransferStateFromPreviousAttempt(),  
9.        appAttemptAddedEvent.getIsAttemptRecovering());  
10.    }  
11.   
12.   
13.   public synchronized void  
14.      addApplicationAttempt(ApplicationAttemptId appAttemptId,  
15.          boolean transferStateFromPreviousAttempt,  
16.          boolean isAttemptRecovering) {  
17.         
18.       //创建SchedulerApplication  
19.    SchedulerApplication<FiCaSchedulerApp> application =  
20.        applications.get(appAttemptId.getApplicationId());  
21.    String user = application.getUser();  
22.    // TODO: Fix store  
23.   
24. //创建FiCaSchedulerApp  
25.    FiCaSchedulerApp schedulerApp =  
26.        new FiCaSchedulerApp(appAttemptId, user, DEFAULT_QUEUE,  
27.          activeUsersManager, this.rmContext);  
28.   
29.    if (transferStateFromPreviousAttempt) {  
30.      schedulerApp.transferStateFromPreviousAttempt(application  
31.        .getCurrentAppAttempt());  
32.    }  
33.   
34. //设置当前attempt  
35.    application.setCurrentAppAttempt(schedulerApp);  
36.   
37. //Submit  
38.    metrics.submitAppAttempt(user);  
39.    LOG.info("Added Application Attempt " + appAttemptId  
40.        + " to scheduler from user " + application.getUser());  
41.    if (isAttemptRecovering) {  
42.      if (LOG.isDebugEnabled()) {  
43.        LOG.debug(appAttemptId  
44.            + " is recovering. Skipping notifying ATTEMPT_ADDED");  
45.      }  
46.    } else {  
47.   
48. //回去触发RMAppAttemptEventType.ATTEMPT_ADDED事件  
49.      rmContext.getDispatcher().getEventHandler().handle(  
50.        new RMAppAttemptEvent(appAttemptId,  
51.            RMAppAttemptEventType.ATTEMPT_ADDED));  
52.    }  
53.  }  

那么接下来就又要回到MRAPPAttemptImpl去看RMAppAttemptEventType.ATTEMPT_ADDED事件了： 

Java代码  

1. .addTransition(RMAppAttemptState.SUBMITTED,   
2.           EnumSet.of(RMAppAttemptState.LAUNCHED_UNMANAGED_SAVING,  
3.                      RMAppAttemptState.SCHEDULED),  
4.           RMAppAttemptEventType.ATTEMPT_ADDED,  
5.           new ScheduleTransition())  

会去执行ScheduleTransition方法， 完成这一步后 这次尝试就会变成scheduled状态， 等着scheduler去assignContainer到nodemanager去了： 

Java代码  

1. public static final class ScheduleTransition  
2.       implements  
3.       MultipleArcTransition<RMAppAttemptImpl, RMAppAttemptEvent, RMAppAttemptState> {  
4.     @Override  
5.     public RMAppAttemptState transition(RMAppAttemptImpl appAttempt,  
6.         RMAppAttemptEvent event) {  
7.           
8.         //这个就是刚刚开始的时候我们创建的包含container启动脚本的地方  
9.       ApplicationSubmissionContext subCtx = appAttempt.submissionContext;  
10.       if (!subCtx.getUnmanagedAM()) {  
11.         // Need reset #containers before create new attempt, because this request  
12.         // will be passed to scheduler, and scheduler will deduct the number after  
13.         // AM container allocated  
14.           
15.         // Currently, following fields are all hard code,  
16.         // TODO: change these fields when we want to support  
17.         // priority/resource-name/relax-locality specification for AM containers  
18.         // allocation.  
19.           
20.         //一个App Master 一个container 设置一些container的属性  
21.         appAttempt.amReq.setNumContainers(1);  
22.         appAttempt.amReq.setPriority(AM_CONTAINER_PRIORITY);  
23.         appAttempt.amReq.setResourceName(ResourceRequest.ANY);  
24.         appAttempt.amReq.setRelaxLocality(true);  
25.           
26.         // AM resource has been checked when submission  
27.           
28.         //去scheduler里面执行allocate 然后会返回一个Allocation对象，  会等NodeManager去heartBeat的时候， ResourceManager发现这个NM还有资源， 然后就assign这个Allocation到这个NM上面， 再去Launch AM  
29.         Allocation amContainerAllocation =  
30.             appAttempt.scheduler.allocate(appAttempt.applicationAttemptId,  
31.                 Collections.singletonList(appAttempt.amReq),  
32.                 EMPTY_CONTAINER_RELEASE_LIST, null, null);  
33.         if (amContainerAllocation != null  
34.             && amContainerAllocation.getContainers() != null) {  
35.           assert (amContainerAllocation.getContainers().size() == 0);  
36.         }  
37.         return RMAppAttemptState.SCHEDULED;  
38.       } else {  
39.         // save state and then go to LAUNCHED state  
40.         appAttempt.storeAttempt();  
41.         return RMAppAttemptState.LAUNCHED_UNMANAGED_SAVING;  
42.       }  
43.     }  
44.   }  

到目前为止AM的Allocation以及生成了， 接下去就是通过Nodemanager去分配container， 然后在NM上面启动Container （执行刚刚开始设置的脚本）