Spark源码精读之《容器分配策略》

一、容器分配是在哪个阶段进行的？

        Spark请求Yarn的容器，不论是Client模式还是Cluster模式，都是走的相同流程。 为了简便，拿Client提交流程标识下。

        下图1，是Spark程序基于Client模式的提交总流程，容器分配就是在第4阶段， 获取Container后进行的。 

对Spark提交流程有兴趣的朋友可以浏览另一篇博文。《Spark源码精读之SparkSubmit》

二、容器分配的策略是？

        容器的分配策略有上中下三策。上策，资源请求和容器在同一个节点；中策，资源请求和容器在相同机架；下策，资源请求和容器任意位置都可。 

        程序先拿请求到的容器，逐一匹配上中下三策。匹配后，剩余用不完的容易，release掉。 然后在筛选到的容器里，启动Exectuor。

三、源码解析

def handleAllocatedContainers(allocatedContainers: Seq[Container]): Unit = {
  val containersToUse = new ArrayBuffer[Container](allocatedContainers.size)

  // Match incoming requests by host 不在同一机器上的容器
  val remainingAfterHostMatches = new ArrayBuffer[Container]
  for (allocatedContainer <- allocatedContainers) {
      matchContainerToRequest(allocatedContainer, allocatedContainer.getNodeId.getHost,containersToUse, remainingAfterHostMatches)
  }
  
  // Match remaining by rack. 不在同一节点上的容器
  val remainingAfterRackMatches = new ArrayBuffer[Container]
  for (allocatedContainer <- remainingAfterHostMatches) {
    val rack = resolver.resolve(allocatedContainer.getNodeId.getHost)
    matchContainerToRequest(allocatedContainer, rack, containersToUse,remainingAfterRackMatches)
  }
  
  // Assign remaining that are neither node-local nor rack-local 将被Release的容器
  val remainingAfterOffRackMatches = new ArrayBuffer[Container]
  for (allocatedContainer <- remainingAfterRackMatches) {
    matchContainerToRequest(allocatedContainer, ANY_HOST, containersToUse,remainingAfterOffRackMatches)
  }
  // 没有用到的容器，release掉。
  if (remainingAfterOffRackMatches.nonEmpty) {
    for (container <- remainingAfterOffRackMatches) {
    internalReleaseContainer(container)
    }
  }
  // 启动容器
  runAllocatedContainers(containersToUse)
}

/**
 * Looks for requests for the given location that match the given container allocation. If it finds one, removes the request so that it won't be submitted again. Places the container into containersToUse or remaining. 查找与给定容器分配匹配的给定位置的请求。如果它找到一个，则删除该请求，这样它就不会再次提交。将容器放入要使用或剩余的容器中。
 * @param allocatedContainer container that was given to us by YARN YARN给的容器中的一个
 * @param location resource name, either a node, rack, or *   资源名称 node、rack或者其他
 * @param containersToUse list of containers that will be used 将被使用的容器列表
 * @param remaining list of containers that will not be used 没被使用的剩余容器
 */
// 这个函数，用于把YARN请求返回的容器和位置进行匹配，匹配成功放入containersToUse参数，否则放入remaining参数。
private def matchContainerToRequest(
    allocatedContainer: Container,
    location: String,
    containersToUse: ArrayBuffer[Container],
    remaining: ArrayBuffer[Container]): Unit = {
    ......
    val matchingRequests = amClient.getMatchingRequests(allocatedContainer.getPriority, location,matchingResource)
    if (!matchingRequests.isEmpty) {
    //若yarn给的容器，和
        val containerRequest = matchingRequests.get(0).iterator.next
        amClient.removeContainerRequest(containerRequest)
        containersToUse += allocatedContainer
    } else {
        remaining += allocatedContainer
    }
}