AppMaster向RM请求资源
//APPMASTER向RM发送心跳,更新资源请求结构,从分配好的内存结构取出分配的资源,具体任务分配是后台异步的由NM发送心跳驱动
MRAppMaster
:serviceinit
// service to allocate containers from RM (if non-uber) or to fake it (uber)
containerAllocator = createContainerAllocator(null, context);
addIfService(containerAllocator);
dispatcher.register(ContainerAllocator.EventType.class, containerAllocator);
protected ContainerAllocator createContainerAllocator(
final ClientService clientService, final AppContext context) {
return new ContainerAllocatorRouter(clientService, context); //
}
private final class ContainerAllocatorRouter extends AbstractService
implements ContainerAllocator, RMHeartbeatHandler {
private final ClientService clientService;
private final AppContext context;
private ContainerAllocator containerAllocator;
.....
@Override
protected void serviceStart() throws Exception {
if (job.isUber()) {
this.containerAllocator = new LocalContainerAllocator(
this.clientService, this.context, nmHost, nmPort, nmHttpPort
, containerID);
} else {
this.containerAllocator = new RMContainerAllocator( ///
this.clientService, this.context);
}
((Service)this.containerAllocator).init(getConfig());
((Service)this.containerAllocator).start();
super.serviceStart();
org.apache.hadoop.mapreduce.v2.app.rm; RMContainerAllocator类有该方法
protected synchronized void heartbeat() throws Exception {
scheduleStats.updateAndLogIfChanged("Before Scheduling: ");
List<Container> allocatedContainers = getResources(); //发远程RM发送心跳信息,注意心跳里可能没有新的资源请求信息
//只是告诉RM自己还活着,或者只是从RM取得分配资源
if (allocatedContainers.size() > 0) {
scheduledRequests.assign(allocatedContainers); //获得的container具体分配到任务task (应该是重排序)
}
资源请求包括的字段:
优先级,期望在的host,内存大小等 (默认三路复制,可能会有7个资源请求,3个local,3个 rack,1个随机)
}
RMContainerAllocator父类RMCommunicator的方法
protected void startAllocatorThread() {
allocatorThread = new Thread(new Runnable() {
@Override
public void run() {
while (!stopped.get() && !Thread.currentThread().isInterrupted()) {
try {
Thread.sleep(rmPollInterval); //默认每秒
try {
heartbeat(); //发送心跳
...
private List<Container> g