本文从源码调用方面介绍从应用程序提交到启动ApplicationMaster的整个过程，期间涉及ClientRMService、RMAppManager、RMAppImpl、RMAppAttemptImpl、RMNode、ResourceScheduler等几个主要组件。
当客户端调用RPC函数ApplicationClientProtocol#submitApplication之后，ResourceManager端的处理过程如下：
步骤1：
ResourceManager中的ClientRMService实现了ApplicationClientProtocol协议，它处理来自客户端的请求，并调用RMAppManager#submitApplication通知其他相关服务作进一步处理。
//clientRMService.java
public SubmitApplicationResponse submitApplication(
SubmitApplicationRequest request) throws YarnException {
...
try {
// call RMAppManager to submit application directly
//开始提交作业
rmAppManager.submitApplication(submissionContext,
System.currentTimeMillis(), user);
LOG.info("Application with id " + applicationId.getId() +
" submitted by user " + user);
RMAuditLogger.logSuccess(user, AuditConstants.SUBMIT_APP_REQUEST,
"ClientRMService", applicationId);
} catch (YarnException e) {
...
}
return response;
}步骤2：
RMAppManager为该应用程序创建一个RMAppImpl对象以维护它的运行状态，并发送一个RMAppEventType.START事件。
//RMAppManager.java
protected void submitApplication(
ApplicationSubmissionContext submissionContext, long submitTime,
String user) throws YarnException {
LOG.info("begin to submitApplication");
//获得作业ID
ApplicationId applicationId = submissionContext.getApplicationId();
//构建一个app并放入applicationACLS
RMAppImpl application =
createAndPopulateNewRMApp(submissionContext, submitTime, user);
ApplicationId appId = submissionContext.getApplicationId();
if (UserGroupInformation.isSecurityEnabled()) {
...
} else {
// Dispatcher is not yet started at this time, so these START events
// enqueued should be guaranteed to be first processed when dispatcher
// gets started.
//触发app启动事件
LOG.info("send event RMAppEventType.START");
LOG.info("this.rmContext="+this.rmContext.toString());
//this.rmContext=RMContextImpl
this.rmContext.getDispatcher().getEventHandler()
.handle(new RMAppEvent(applicationId, RMAppEventType.START));
}
}步骤3：
RMAppImpl收到RMAppEventType.START事件后，会调用RMStateStore#storeApplication，以日志记录RMAppImpl当前信息，
至此，RMAppImpl的运行状态由NEW转移为NEW_SAVING。该步骤就较为复杂了，下面详细介绍下。
其中RMAppEventType注册到中央异步调度器的地方在ResourceManager.java中：
//ResourceManager.java
protected void serviceInit(Configuration configuration) throws Exception {
conf.setBoolean(Dispatcher.DISPATCHER_EXIT_ON_ERROR_KEY, true);
...
rmDispatcher.register(SchedulerEventType.class, schedulerDispatcher);
// Register event handler for RmAppEvents
rmDispatcher.register(RMAppEventType.class,
new ApplicationEventDispatcher(rmContext));
// Register event handler for RmAppAttemptEvents
rmDispatcher.register(RMAppAttemptEventType.class,
new ApplicationAttemptEventDispatcher(rmContext));
// Register event handler for RmNodes
rmDispatcher.register(
RMNodeEventType.class, new NodeEventDispatcher(rmContext));
...
}上面的this.rmContext=RMContextImpl，
this.rmContext.getDispatcher()=AsyncDispatcher，
this.rmContext.getDispatcher().getEventHandler()=AsyncDispatcher$GenericEventHandler
所以会进入AsyncDispatcher类中的内部类GenericEventHandler的函数handle中
//AsyncDispatcher.java
class GenericEventHandler implements EventHandler<Event> {
public void handle(Event event) {
LOG.info("begin to call GenericEventHandler::handle, event= "+event.toString());
if (blockNewEvents) {
return;
}
drained = false;
/* all this method does is enqueue all the events onto the queue */
int qSize = eventQueue.size();
if (qSize !=0 && qSize %1000 == 0) {
LOG.info("Size of event-queue is " + qSize);
}
int remCapacity = eventQueue.remainingCapacity();
if (remCapacity < 1000) {
LOG.warn("Very low remaining capacity in the event-queue: "
+ remCapacity);
}
try {
LOG.info("begin to put event in queue.");
eventQueue.put(event);
} catch (InterruptedException e) {
if (!stopped) {
LOG.warn("AsyncDispatcher thread interrupted", e);
}
throw new YarnRuntimeException(e);
}
};
}注意这个异步调度器AsyncDispatcher类是公用的。
RMAppEventType.START事件放入队列eventQueue中，会被RMAppImpl类获取，进入其handle函数
//RMAppImpl.java
public void handle(RMAppEvent event) {
this.writeLock.lock();
try {
ApplicationId appID = event.getApplicationId();
LOG.info("Processing event for " + appID + " of type "
+ event.getType());
final RMAppState oldState = getState();
try {
/* keep the master in sync with the state machine */
this.stateMachine.doTransition(event.getType(), event);
} catch (InvalidStateTransitonException e) {
LOG.error("Can't handle this event at current state", e);
/* fail the application on the failed transition */
}
if (oldState != getState()) {
LOG.info(appID + " State change from " + oldState + " to "
+ getState());
}
} finally {
this.writeLock.unlock();
}
}这里面的关键语句是
this.stateMachine.doTransition(event.getType(), event);
这个stateMachine是个状态机工厂，其中绑定了很多的事件转换：
//RMAppImpl.java
private static final StateMachineFactory<RMAppImpl,
RMAppState,
RMAppEventType,
RMAppEvent> stateMachineFactory
= new StateMachineFactory<RMAppImpl,
RMAppState,
RMAppEventType,
RMAppEvent>(RMAppState.NEW)
// Transitions from NEW state
.addTransition(RMAppState.NEW, RMAppState.NEW,
RMAppEventType.NODE_UPDATE, new RMAppNodeUpdateTransition())
.addTransition(RMAppState.NEW, RMAppState.NEW_SAVING,
RMAppEventType.START, new RMAppNewlySavingTransition())
.addTransition(...)
...
addTransition(...)addTransition(RMAppState.NEW, RMAppState.NEW_SAVING, RMAppEventType.START, new RMAppNewlySavingTransition())
意思就是接受RMAppEventType.START类型的事件，将状态由RMAppState.NEW转换为RMAppState.NEW_SAVING，调用的回调类是RMAppNewlySavingTransition。
在addTransition函数中，就将第二个参数postState传给了新构建的内部类SingleInternalArc
//StateMachineFactory.java
public StateMachineFactory
<OPERAND, STATE, EVENTTYPE, EVENT>
addTransition(STATE preState, STATE postState,
EVENTTYPE eventType,
SingleArcTransition<OPERAND, EVENT> hook){
return new StateMachineFactory<OPERAND, STATE, EVENTTYPE, EVENT>
(this, new ApplicableSingleOrMultipleTransition<OPERAND, STATE, EVENTTYPE, EVENT>
(preState, eventType, new SingleInternalArc(postState, hook)));
}初始化的内部类SingleInternalArc中，
保存了状态转换之后的值postState，此时的值就是RMAppState.NEW_SAVING。
也保存了回调函数hook=RMAppNewlySavingTransition。
//StateMachineFactory.java
SingleInternalArc(STATE postState,
SingleArcTransition<OPERAND, EVENT> hook) {
this.postState = postState;
this.hook = hook;
}返回到RMAppImpl类的handle函数中，调用this.stateMachine.doTransition(event.getType(), event)：
进入到StateMachineFactory类中
//StateMachineFactory.java
public synchronized STATE doTransition(EVENTTYPE eventType, EVENT event)
throws InvalidStateTransitonException {
LOG.info("StateMachineFactory::InternalStateMachine::doTransition 1111");
currentState = StateMachineFactory.this.doTransition
(operand, currentState, eventType, event);
return currentState;
}再调用到
//StateMachineFactory.java
private STATE doTransition
(OPERAND operand, STATE oldState, EVENTTYPE eventType, EVENT event)
throws InvalidStateTransitonException {
LOG.info("StateMachineFactory::doTransition 2222");
LOG.info("operand="+operand.toString()+",oldState="+oldState.toString());
// We can assume that stateMachineTable is non-null because we call
// maybeMakeStateMachineTable() when we build an InnerStateMachine ,
// and this code only gets called from inside a working InnerStateMachine .
Map<EVENTTYPE, Transition<OPERAND, STATE, EVENTTYPE, EVENT>> transitionMap
= stateMachineTable.get(oldState);
if (transitionMap != null) {
Transition<OPERAND, STATE, EVENTTYPE, EVENT> transition
= transitionMap.get(eventType);
if (transition != null) {
LOG.info("transition="+transition.toString());
return transition.doTransition(operand, oldState, event, eventType);
}
}
throw new InvalidStateTransitonException(oldState, eventType);
}到return transition.doTransition(operand, oldState, event, eventType);
其中oldState=NEW，
transition=org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc@3eca92d3
进入内部类SingleInternalArc的函数doTransition中
//StateMachineFactory.java
private class SingleInternalArc
implements Transition<OPERAND, STATE, EVENTTYPE, EVENT> {
...
@Override
public STATE doTransition(OPERAND operand, STATE oldState,
EVENT event, EVENTTYPE eventType) {
if (hook != null) {
LOG.info("SingleInternalArc::doTransition, hook = "+hook.toString());
hook.transition(operand, event);
}
return postState;
}
}返回值return postState;就是上面保存的RMAppState.NEW_SAVING，到上层保存在变量currentState里，这个变量在RMAppImpl中被get函数getState()获取，
//RMAppImpl.java
if (oldState != getState()) {
LOG.info(appID + " State change from " + oldState + " to "
+ getState());
}2017-02-20 22:59:07,702 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: application_1487602693580_0001 State change from NEW to NEW_SAVING
此时就完成了一次状态机的状态转变。
再回到hook变量，hook初始化的就是上面状态机绑定的回调类RMAppNewlySavingTransition。
再次回调到了RMAppImpl类中的内部类RMAppNewlySavingTransition的transition函数里。
//RMAppImpl.java
private static final class RMAppNewlySavingTransition extends RMAppTransition {
@Override
public void transition(RMAppImpl app, RMAppEvent event) {
// If recovery is enabled then store the application information in a
// non-blocking call so make sure that RM has stored the information
// needed to restart the AM after RM restart without further client
// communication
LOG.info("RMAppNewlySavingTransition::transition Storing application with id " + app.applicationId);
//app.rmContext=RMContextImpl
LOG.info("RMAppNewlySavingTransition::transition app.rmContext="+app.rmContext.toString());
//app.rmContext.getStateStore()=RMStateStore
LOG.info("RMAppNewlySavingTransition::transition app.rmContext.getStateStore()="+app.rmContext.getStateStore().toString());
app.rmContext.getStateStore().storeNewApplication(app);
}
}其中，
app.rmContext=RMContextImpl
app.rmContext.getStateStore()=RMStateStore
进入RMStateStore类的storeNewApplication函数里
//RMStateStore.java
public synchronized void storeNewApplication(RMApp app) {
//app=RMAppImpl
LOG.info("begin to storeNewApplication,app="+app.toString());
ApplicationSubmissionContext context = app.getApplicationSubmissionContext();
assert context instanceof ApplicationSubmissionContextPBImpl;
ApplicationState appState =
new ApplicationState(app.getSubmitTime(), app.getStartTime(), context,app.getUser());
dispatcher.getEventHandler().handle(new RMStateStoreAppEvent(appState));
}//RMStateStoreAppEvent.java
public RMStateStoreAppEvent(ApplicationState appState) {
super(RMStateStoreEventType.STORE_APP);
this.appState = appState;
}此处的dispatcher是RMStateStore类自己的变量，只在初始化时绑定了一个RMStateStoreEventType：
//RMStateStore.java
protected void serviceInit(Configuration conf) throws Exception{
// create async handler
dispatcher = new AsyncDispatcher();
dispatcher.init(conf);
dispatcher.register(RMStateStoreEventType.class, new ForwardingEventHandler());
dispatcher.setDrainEventsOnStop();
initInternal(conf);
}//RMStateStore.java
private final class ForwardingEventHandler implements EventHandler<RMStateStoreEvent> {
@Override
public void handle(RMStateStoreEvent event) {
LOG.info("ForwardingEventHandler::handle ");
handleStoreEvent(event);
}
}其中的handle函数调用了函数handleStoreEvent：
//RMStateStore.java
protected void handleStoreEvent(RMStateStoreEvent event) {
try {
LOG.info("handleStoreEvent,event.getType()="+event.getType().toString()+", event="+event.toString());
this.stateMachine.doTransition(event.getType(), event);
} catch (InvalidStateTransitonException e) {
LOG.error("Can't handle this event at current state", e);
}
}又会进入类StateMachineFactory中的内部类SingleInternalArc里，
不过这次的状态机工厂是RMStateStore类的内部变量，上面的状态机工厂是RMAppImpl类的，他们绑定的事件不同。
//RMStateStore.java
private static final StateMachineFactory<RMStateStore,
RMStateStoreState,
RMStateStoreEventType,
RMStateStoreEvent>
stateMachineFactory = new StateMachineFactory<RMStateStore,
RMStateStoreState,
RMStateStoreEventType,
RMStateStoreEvent>(
RMStateStoreState.DEFAULT)
.addTransition(RMStateStoreState.DEFAULT, RMStateStoreState.DEFAULT,
RMStateStoreEventType.STORE_APP, new StoreAppTransition())
.addTransition(RMStateStoreState.DEFAULT, RMStateStoreState.DEFAULT,
RMStateStoreEventType.UPDATE_APP, new UpdateAppTransition())
.addTransition(RMStateStoreState.DEFAULT, RMStateStoreState.DEFAULT,
RMStateStoreEventType.REMOVE_APP, new RemoveAppTransition())
.addTransition(RMStateStoreState.DEFAULT, RMStateStoreState.DEFAULT,
RMStateStoreEventType.STORE_APP_ATTEMPT, new StoreAppAttemptTransition())
.addTransition(RMStateStoreState.DEFAULT, RMStateStoreState.DEFAULT,
RMStateStoreEventType.UPDATE_APP_ATTEMPT, new UpdateAppAttemptTransition());RMStateStoreEventType.STORE_APP事件只是将状态RMStateStoreState.DEFAULT再转变为自身。RMStateStoreState本事也只有一个状态DEFAULT。
主要作用就是完成RMAppImpl类当前信息的日志记录。日记记录是为了RM的重启，详情见ResourceManager重启过程。
RMStateStoreEventType.STORE_APP事件绑定的类是StoreAppTransition，在StateMachineFactory中的内部类SingleInternalArc里调用hook.transition(operand, event);
就会回到RMStateStore中的内部类StoreAppTransition里的函数transition中。
//RMStateStore.java
private static class StoreAppTransition
implements SingleArcTransition<RMStateStore, RMStateStoreEvent> {
@Override
public void transition(RMStateStore store, RMStateStoreEvent event) {
LOG.info("StoreAppTransition :: transition,event="+event.toString());
if (!(event instanceof RMStateStoreAppEvent)) {
// should never happen
LOG.error("Illegal event type: " + event.getClass());
return;
}
ApplicationState appState = ((RMStateStoreAppEvent) event).getAppState();
ApplicationId appId = appState.getAppId();
ApplicationStateData appStateData = ApplicationStateData
.newInstance(appState);
LOG.info("Storing info for app: " + appId);
try {
store.storeApplicationStateInternal(appId, appStateData);
store.notifyDoneStoringApplication(appId, null);
} catch (Exception e) {
LOG.error("Error storing app: " + appId, e);
store.notifyStoreOperationFailed(e);
}
};
}步骤4：
日志记录完成后，RMStateStore进一步向RMAppImpl发送RMAppEventType.APP_NEW_SAVED事件。
上一步中，调用 store.notifyDoneStoringApplication(appId, null);
//RMStateStore.java
private void notifyDoneStoringApplication(ApplicationId appId,Exception storedException) {
LOG.info("notifyDoneStoringApplication, send RMAppEventType.APP_NEW_SAVED");
rmDispatcher.getEventHandler().handle(new RMAppNewSavedEvent(appId, storedException));
}//RMAppNewSavedEvent.java
public RMAppNewSavedEvent(ApplicationId appId, Exception storedException) {
super(appId, RMAppEventType.APP_NEW_SAVED);
this.storedException = storedException;
}事件类型是RMAppEventType.APP_NEW_SAVED，
由于在ResourceManager中，将RMAppEventType类型的事件绑定到了ApplicationEventDispatcher中：
//ResourceManager.java
// Register event handler for RmAppEvents
rmDispatcher.register(RMAppEventType.class,new ApplicationEventDispatcher(rmContext));步骤5：
RMAppImpl收到RMAppEventType.APP_NEW_SAVED事件后，将自身的运行状态由NEW_SAVING转换为SUBMITTED ，同时向scheduler发送一个SchedulerEventType.APP_ADDED事件，
//RMAppImpl.java
.addTransition(RMAppState.NEW_SAVING, RMAppState.SUBMITTED,
RMAppEventType.APP_NEW_SAVED, new AddApplicationToSchedulerTransition())//RMAppImpl.java
private static final class AddApplicationToSchedulerTransition extends
RMAppTransition {
@Override
public void transition(RMAppImpl app, RMAppEvent event) {
if (event instanceof RMAppNewSavedEvent) {
LOG.info("AddApplicationToSchedulerTransition::transition event="+event.toString());
RMAppNewSavedEvent storeEvent = (RMAppNewSavedEvent) event;
// For HA this exception needs to be handled by giving up
// master status if we got fenced
if (((RMAppNewSavedEvent) event).getStoredException() != null) {
LOG.error(
"Failed to store application: " + storeEvent.getApplicationId(),
storeEvent.getStoredException());
ExitUtil.terminate(1, storeEvent.getStoredException());
}
}
app.handler.handle(new AppAddedSchedulerEvent(app.applicationId,
app.submissionContext.getQueue(), app.user));
}
}//AppAddedSchedulerEvent.java
public AppAddedSchedulerEvent(
ApplicationId applicationId, String queue, String user) {
super(SchedulerEventType.APP_ADDED);
this.applicationId = applicationId;
this.queue = queue;
this.user = user;
}步骤6：
scheduler收到SchedulerEventType.APP_ADDED事件之后，首先进行权限检查，然后将应用程序信息保存到内部的数据结构中，
并向RMAppImpl发送APP_ACCEPTED事件。
在ResourceManager中，SchedulerEventType事件类注册到了schedulerDispatcher类中：
//ResourceManager.java
schedulerDispatcher = createSchedulerEventDispatcher();
addIfService(schedulerDispatcher);
rmDispatcher.register(SchedulerEventType.class, schedulerDispatcher);//ResourceManager.java
protected EventHandler<SchedulerEvent> createSchedulerEventDispatcher() {
return new SchedulerEventDispatcher(this.scheduler);
}//ResourceManager.java
// Initialize the scheduler
scheduler = createScheduler();
scheduler.setRMContext(rmContext);
addIfService(scheduler);
rmContext.setScheduler(scheduler);//ResourceManager.java
protected ResourceScheduler createScheduler() {
String schedulerClassName = conf.get(YarnConfiguration.RM_SCHEDULER,
YarnConfiguration.DEFAULT_RM_SCHEDULER);
LOG.info("Using Scheduler: " + schedulerClassName);
try {
Class<?> schedulerClazz = Class.forName(schedulerClassName);
if (ResourceScheduler.class.isAssignableFrom(schedulerClazz)) {
return (ResourceScheduler) ReflectionUtils.newInstance(schedulerClazz,
this.conf);
} else {
throw new YarnRuntimeException("Class: " + schedulerClassName
+ " not instance of " + ResourceScheduler.class.getCanonicalName());
}
} catch (ClassNotFoundException e) {
throw new YarnRuntimeException("Could not instantiate Scheduler: "
+ schedulerClassName, e);
}
}取值为org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler
进入CapacityScheduler类中
//CapacityScheduler.java
public void handle(SchedulerEvent event) {
switch(event.getType()) {
...
case APP_ADDED:
{
AppAddedSchedulerEvent appAddedEvent = (AppAddedSchedulerEvent) event;
addApplication(appAddedEvent.getApplicationId(),
appAddedEvent.getQueue(), appAddedEvent.getUser());
}
break;
...
}//CapacityScheduler.java
private synchronized void addApplication(ApplicationId applicationId,
String queueName, String user) {
LOG.info("begin to call addApplication");
// santiy checks.
CSQueue queue = getQueue(queueName);
if (queue == null) {
String message = "Application " + applicationId +
" submitted by user " + user + " to unknown queue: " + queueName;
this.rmContext.getDispatcher().getEventHandler()
.handle(new RMAppRejectedEvent(applicationId, message));
return;
}
if (!(queue instanceof LeafQueue)) {
String message = "Application " + applicationId +
" submitted by user " + user + " to non-leaf queue: " + queueName;
this.rmContext.getDispatcher().getEventHandler()
.handle(new RMAppRejectedEvent(applicationId, message));
return;
}
// Submit to the queue
try {
queue.submitApplication(applicationId, user, queueName);
} catch (AccessControlException ace) {
LOG.info("Failed to submit application " + applicationId + " to queue "
+ queueName + " from user " + user, ace);
this.rmContext.getDispatcher().getEventHandler()
.handle(new RMAppRejectedEvent(applicationId, ace.toString()));
return;
}
SchedulerApplication<FiCaSchedulerApp> application =
new SchedulerApplication<FiCaSchedulerApp>(queue, user);
applications.put(applicationId, application);
LOG.info("Accepted application " + applicationId + " from user: " + user
+ ", in queue: " + queueName);
rmContext.getDispatcher().getEventHandler()
.handle(new RMAppEvent(applicationId, RMAppEventType.APP_ACCEPTED));
}最后向RMAppImpl发送了RMAppEventType.APP_ACCEPTED事件。
步骤7：
RMAppImpl接受到RMAppEventType.APP_ACCEPTED事件后，将状态从SUBMITTED转换到ACCEPTED。
//RMAppImpl.java
.addTransition(RMAppState.SUBMITTED, RMAppState.ACCEPTED,
RMAppEventType.APP_ACCEPTED, new StartAppAttemptTransition())//RMAppImpl.java
private static final class StartAppAttemptTransition extends RMAppTransition {
@Override
public void transition(RMAppImpl app, RMAppEvent event) {
LOG.info("class::StartAppAttemptTransition, func::transition, ");
app.createAndStartNewAttempt(false);
};
}//RMAppImpl.java
private void
createAndStartNewAttempt(boolean transferStateFromPreviousAttempt) {
LOG.info("func::createAndStartNewAttempt,begin ");
createNewAttempt();
handler.handle(new RMAppStartAttemptEvent(currentAttempt.getAppAttemptId(),
transferStateFromPreviousAttempt));
}
//RMAppStartAttemptEvent.java
public RMAppStartAttemptEvent(ApplicationAttemptId appAttemptId,
boolean transferStateFromPreviousAttempt) {
super(appAttemptId, RMAppAttemptEventType.START);
this.transferStateFromPreviousAttempt = transferStateFromPreviousAttempt;
}
RMAppAttemptImpl接受RMAppAttemptEventType.START事件后，进行一系列初始化工作。
在类RMAppAttemptImpl中，也有状态机工厂：
//RMAppAttemptImpl.java
.addTransition(RMAppAttemptState.NEW, RMAppAttemptState.SUBMITTED,
RMAppAttemptEventType.START, new AttemptStartedTransition())//RMAppAttemptImpl.java
private static final class AttemptStartedTransition extends BaseTransition {
@Override
public void transition(RMAppAttemptImpl appAttempt,
RMAppAttemptEvent event) {
LOG.info("class::AttemptStartedTransition, func::transition, begin. ");
boolean transferStateFromPreviousAttempt = false;
if (event instanceof RMAppStartAttemptEvent) {
transferStateFromPreviousAttempt =
((RMAppStartAttemptEvent) event)
.getTransferStateFromPreviousAttempt();
}
appAttempt.startTime = System.currentTimeMillis();
// Register with the ApplicationMasterService
appAttempt.masterService
.registerAppAttempt(appAttempt.applicationAttemptId);
if (UserGroupInformation.isSecurityEnabled()) {
appAttempt.clientTokenMasterKey =
appAttempt.rmContext.getClientToAMTokenSecretManager()
.createMasterKey(appAttempt.applicationAttemptId);
}
// create AMRMToken
AMRMTokenIdentifier id =
new AMRMTokenIdentifier(appAttempt.applicationAttemptId);
appAttempt.amrmToken =
new Token<AMRMTokenIdentifier>(id,
appAttempt.rmContext.getAMRMTokenSecretManager());
// Add the applicationAttempt to the scheduler and inform the scheduler
// whether to transfer the state from previous attempt.
LOG.info("send SchedulerEventType.APP_ATTEMPT_ADDED");
appAttempt.eventHandler.handle(new AppAttemptAddedSchedulerEvent(
appAttempt.applicationAttemptId, transferStateFromPreviousAttempt));
}
}步骤9：
CapacityScheduler收到SchedulerEventType.APP_ATTEMPT_ADDED事件后，
走到默认调度器CapacityScheduler中的handle函数：
//CapacityScheduler.java
public void handle(SchedulerEvent event) {
switch(event.getType()) {
...
case APP_ATTEMPT_ADDED:
{
AppAttemptAddedSchedulerEvent appAttemptAddedEvent =
(AppAttemptAddedSchedulerEvent) event;
addApplicationAttempt(appAttemptAddedEvent.getApplicationAttemptId(),
appAttemptAddedEvent.getTransferStateFromPreviousAttempt(),
appAttemptAddedEvent.getShouldNotifyAttemptAdded());
}
break;
...
}
//CapacityScheduler.java
private synchronized void addApplicationAttempt(
ApplicationAttemptId applicationAttemptId,
boolean transferStateFromPreviousAttempt,
boolean shouldNotifyAttemptAdded) {
LOG.info("func::addApplicationAttempt, begin.");
SchedulerApplication<FiCaSchedulerApp> application =
applications.get(applicationAttemptId.getApplicationId());
CSQueue queue = (CSQueue) application.getQueue();
FiCaSchedulerApp attempt =
new FiCaSchedulerApp(applicationAttemptId, application.getUser(),
queue, queue.getActiveUsersManager(), rmContext);
if (transferStateFromPreviousAttempt) {
attempt.transferStateFromPreviousAttempt(application
.getCurrentAppAttempt());
}
application.setCurrentAppAttempt(attempt);
queue.submitApplicationAttempt(attempt, application.getUser());
LOG.info("Added Application Attempt " + applicationAttemptId
+ " to scheduler from user " + application.getUser() + " in queue "
+ queue.getQueueName());
if (shouldNotifyAttemptAdded) {
rmContext.getDispatcher().getEventHandler().handle(
new RMAppAttemptEvent(applicationAttemptId,
RMAppAttemptEventType.ATTEMPT_ADDED));
} else {
if (LOG.isDebugEnabled()) {
LOG.debug("Skipping notifying ATTEMPT_ADDED");
}
}
}将运行实例加入到队列中：
queue.submitApplicationAttempt(attempt, application.getUser());Added Application Attempt appattempt_1487944669971_0001_000001 to scheduler from user root in queue default
最后发送事件RMAppAttemptEventType.ATTEMPT_ADDED给RMAppAttemptImpl。
步骤10：
RMAppAttemptImpl接受到事件RMAppAttemptEventType.ATTEMPT_ADDED后，
//RMAppAttemptImpl.java
.addTransition(RMAppAttemptState.SUBMITTED,
EnumSet.of(RMAppAttemptState.LAUNCHED_UNMANAGED_SAVING,
RMAppAttemptState.SCHEDULED),
RMAppAttemptEventType.ATTEMPT_ADDED,
new ScheduleTransition())进入内部类ScheduleTransition的transition函数：
//RMAppAttemptImpl.java
private static final class ScheduleTransition
implements
MultipleArcTransition<RMAppAttemptImpl, RMAppAttemptEvent, RMAppAttemptState> {
@Override
public RMAppAttemptState transition(RMAppAttemptImpl appAttempt,
RMAppAttemptEvent event) {
LOG.info("class::ScheduleTransition, func::transition, begin.");
if (!appAttempt.submissionContext.getUnmanagedAM()) {
// Request a container for the AM.
ResourceRequest request =
BuilderUtils.newResourceRequest(
AM_CONTAINER_PRIORITY, ResourceRequest.ANY, appAttempt
.getSubmissionContext().getResource(), 1);
// SchedulerUtils.validateResourceRequests is not necessary because
// AM resource has been checked when submission
Allocation amContainerAllocation = appAttempt.scheduler.allocate(
appAttempt.applicationAttemptId,
Collections.singletonList(request), EMPTY_CONTAINER_RELEASE_LIST, null, null);
if (amContainerAllocation != null
&& amContainerAllocation.getContainers() != null) {
assert (amContainerAllocation.getContainers().size() == 0);
}
return RMAppAttemptState.SCHEDULED;
} else {
// save state and then go to LAUNCHED state
appAttempt.storeAttempt();
return RMAppAttemptState.LAUNCHED_UNMANAGED_SAVING;
}
}
}
为AM申请container资源，该资源描述如下：
ResourceRequest request =
BuilderUtils.newResourceRequest(
AM_CONTAINER_PRIORITY, ResourceRequest.ANY, appAttempt
.getSubmissionContext().getResource(), 1);资源量为X（appAttempt .getSubmissionContext().getResource()，用户提交应用程序时指定的）的Container。
步骤11：
ResourceManager为应用程序的AM分配资源后，创建一个RMContainerImpl，并向它发送一个RMContainerEventType.START事件。
步骤12：
RMContainerImpl收到RMContainerEventType.START事件后，直接向RMAppAttemptImpl发送一个RMAppAttemptEventType.CONTAINER_ALLOCATED
事件，至此，RMContainerImpl的状态由NEW转换为ALLOCATED。
步骤13：
RMAppAttemptImpl收到RMAppAttemptEventType.CONTAINER_ALLOCATED事件后：
//RMAppAttemptImpl.java
.addTransition(RMAppAttemptState.SCHEDULED,
EnumSet.of(RMAppAttemptState.ALLOCATED_SAVING,
RMAppAttemptState.SCHEDULED),
RMAppAttemptEventType.CONTAINER_ALLOCATED,
new AMContainerAllocatedTransition())//RMAppAttemptImpl.java
private static final class AMContainerAllocatedTransition
implements
MultipleArcTransition<RMAppAttemptImpl, RMAppAttemptEvent, RMAppAttemptState> {
@Override
public RMAppAttemptState transition(RMAppAttemptImpl appAttempt,
RMAppAttemptEvent event) {
// Acquire the AM container from the scheduler.
Allocation amContainerAllocation =
appAttempt.scheduler.allocate(appAttempt.applicationAttemptId,
EMPTY_CONTAINER_REQUEST_LIST, EMPTY_CONTAINER_RELEASE_LIST, null,
null);
// There must be at least one container allocated, because a
// CONTAINER_ALLOCATED is emitted after an RMContainer is constructed,
// and is put in SchedulerApplication#newlyAllocatedContainers.
// Note that YarnScheduler#allocate is not guaranteed to be able to
// fetch it since container may not be fetchable for some reason like
// DNS unavailable causing container token not generated. As such, we
// return to the previous state and keep retry until am container is
// fetched.
if (amContainerAllocation.getContainers().size() == 0) {
appAttempt.retryFetchingAMContainer(appAttempt);
return RMAppAttemptState.SCHEDULED;
}
// Set the masterContainer
appAttempt.setMasterContainer(amContainerAllocation.getContainers()
.get(0));
RMContainerImpl rmMasterContainer = (RMContainerImpl)appAttempt.scheduler
.getRMContainer(appAttempt.getMasterContainer().getId());
rmMasterContainer.setAMContainer(true);
// The node set in NMTokenSecrentManager is used for marking whether the
// NMToken has been issued for this node to the AM.
// When AM container was allocated to RM itself, the node which allocates
// this AM container was marked as the NMToken already sent. Thus,
// clear this node set so that the following allocate requests from AM are
// able to retrieve the corresponding NMToken.
appAttempt.rmContext.getNMTokenSecretManager()
.clearNodeSetForAttempt(appAttempt.applicationAttemptId);
appAttempt.getSubmissionContext().setResource(
appAttempt.getMasterContainer().getResource());
appAttempt.storeAttempt();
return RMAppAttemptState.ALLOCATED_SAVING;
}
}
transition函数中，调用scheduler.allocate获取分配的资源，scheduler返回资源之前，会向RMContainerImpl发送
RMContainerEventType.ALLOCATED事件。
RMAppAttemptImpl收到资源后，向RMStateStore发送MStateStoreEventType.STORE_APP_ATTEMPT事件请求记录日志。
至此，RMAppAttemptImpl状态从SCHEDULED 转换为 ALLOCATED_SAVING。
步骤14：
日志记录完成后，RMStateStore向RMAppAttemptImpl发送RMAppAttemptEventType.ATTEMPT_NEW_SAVED事件。
步骤15：
RMAppAttemptImpl收到RMAppAttemptEventType.ATTEMPT_NEW_SAVED事件后，向ApplicationMasterLauncher
发送AMLauncherEventType.LAUNCH事件，至此，RMAppAttemptImpl状态从ALLOCATED_SAVING转换为ALLOCATED。
步骤16：
ApplicationMasterLauncher收到AMLauncherEventType.LAUNCH事件后，
//ApplicationMasterLauncher.java
public synchronized void handle(AMLauncherEvent appEvent) {
AMLauncherEventType event = appEvent.getType();
RMAppAttempt application = appEvent.getAppAttempt();
switch (event) {
case LAUNCH:
launch(application);
break;
case CLEANUP:
cleanup(application);
default:
break;
}
}//ApplicationMasterLauncher.java
private void launch(RMAppAttempt application) {
Runnable launcher = createRunnableLauncher(application,
AMLauncherEventType.LAUNCH);
masterEvents.add(launcher);
}调用launch函数，将该事件放入事件队列masterEvents中，
//ApplicationMasterLauncher.java
private final BlockingQueue<Runnable> masterEvents
= new LinkedBlockingQueue<Runnable>();等待AMLauncher线程池中的线程处理该事件
//ApplicationMasterLauncher.java
private class LauncherThread extends Thread {
public LauncherThread() {
super("ApplicationMaster Launcher");
}
@Override
public void run() {
while (!this.isInterrupted()) {
Runnable toLaunch;
try {
toLaunch = masterEvents.take();
launcherPool.execute(toLaunch);
} catch (InterruptedException e) {
LOG.warn(this.getClass().getName() + " interrupted. Returning.");
return;
}
}
}
} RMAppAttemptImpl收到RMAppAttemptEventType.LAUNCHED事件后，会向AMLivelinessMonitor注册，以监控运行状态：
//RMAppAttemptImpl.java
private static class AMLaunchedTransition extends BaseTransition {
@Override
public void transition(RMAppAttemptImpl appAttempt,
RMAppAttemptEvent event) {
// Register with AMLivelinessMonitor
appAttempt.attemptLaunched();
// register the ClientTokenMasterKey after it is saved in the store,
// otherwise client may hold an invalid ClientToken after RM restarts.
appAttempt.rmContext.getClientToAMTokenSecretManager()
.registerApplication(appAttempt.getAppAttemptId(),
appAttempt.getClientTokenMasterKey());
}
}至此，RMAppAttemptImpl的状态从ALLOCATED转换为LAUNCHED。
步骤17：
NodeManager通过心跳机制汇报AM所在Container已经成功启动，收到该消息后，scheduler将发送一个RMContainerEventType.LAUNCHED
事件，RMContainerImpl收到该事件后，会从containerAllocationExpirer监控列表中移除。
//RMContainerImpl.java
// Transitions from ACQUIRED state
.addTransition(RMContainerState.ACQUIRED, RMContainerState.RUNNING,
RMContainerEventType.LAUNCHED, new LaunchedTransition())//RMContainerImpl.java
private static final class LaunchedTransition extends BaseTransition {
@Override
public void transition(RMContainerImpl container, RMContainerEvent event) {
// Unregister from containerAllocationExpirer.
container.containerAllocationExpirer.unregister(container
.getContainerId());
}
}步骤18：
启动的AM通过RPC函数ApplicationMasterProtocol#registerApplicationMaster向RM注册，RM中的ApplicationMasterService服务
接收到该请求后，将向RMAppAttemptImpl发送一个RMAppAttemptEventType.REGISTERED事件，而RMAppAttemptImpl收到该事件后，
首先保存该AM的基本信息，然后向RMAppImpl发送RMAppEventType.ATTEMPT_REGISTERED，
至此，RMAppAttemptImpl状态从LAUNCHED转换为LAUNCHED。
步骤19：
RMAppImpl接受到RMAppEventType.ATTEMPT_REGISTERED后，只将状态从ACCEPTED转换为RUNNING：
//RMAppImpl.java
.addTransition(RMAppState.ACCEPTED, RMAppState.RUNNING,
RMAppEventType.ATTEMPT_REGISTERED)
完。
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
