ccah-500 第5题 How will the Fair Scheduler handle these two jobs?

原创 2016年06月01日 10:11:44

5.You have a cluster running with the fair Scheduler enabled. There are currently no jobs running on the cluster, and you submit a job A, so that only job A is running on the cluster. A while later, you submit Job B. now Job A and Job B are running on the cluster at the same time. How will the Fair Scheduler handle these two jobs? 

A. When Job B gets submitted, it will get assigned tasks, while job A continues to run with fewer tasks. 

B. When Job B gets submitted, Job A has to finish first, before job B can gets scheduled. 

C. When Job A gets submitted, it doesn't consumes all the task slots. 

D. When Job A gets submitted, it consumes all the task slots. 

Answer: B --> A

解析: A



With the Fair Scheduler (iii in Figure 4-3), there is no need to reserve a set amount of

capacity, since it will dynamically balance resources between all running jobs. Just after

the first (large) job starts, it is the only job running, so it gets all the resources in the

cluster. When the second (small) job starts, it is allocated half of the cluster resources so

that each job is using its fair share of resources.

Note that there is a lag between the time the second job starts and when it receives its fair

share, since it has to wait for resources to free up as containers used by the first job

complete. After the small job completes and no longer requires resources, the large job

goes back to using the full cluster capacity again. The overall effect is both high cluster

utilization and timely small job completion.


Fair scheduling is a method of assigning resources to jobs such that all jobs get, on average, an equal share of resources over time. When there is a single job running, that job uses the entire cluster. When other jobs are submitted, tasks slots that free up are assigned to the new jobs, so that each job gets roughly the same amount of CPU time. Unlike the default Hadoop scheduler, which forms a queue of jobs, this lets short jobs finish in reasonable time while not starving long jobs. It is also a reasonable way to share a cluster between a number of users.



解决Intellij idea Java JDK多重选择提示问题

问题引出 当前我们对idea 写Java的程序进行编译时,会报如下的错误提示,原因在于 idea 检测到了两个位置有jdk,它不知道选哪一个,就随便选了一个。 objc[63766]: Class...
  • ruglcc
  • ruglcc
  • 2017年05月22日 16:14
  • 3779

Qt学习(六) VS2013编译Qt5.3.1的静态库

  • wwkaven
  • wwkaven
  • 2014年07月11日 08:36
  • 11022


算法与设计分析作业3贪心Greedy Algorithm Pseudo-code Prove the correctness The complexity of your algorithm Gree...
  • King_Like_Coding
  • King_Like_Coding
  • 2017年01月09日 20:00
  • 1295

ccah-500 第57题 Which two best describes how FIFO Scheduler arbitrates the cluster resources for job

57.You have a cluster running with a FIFO scheduler enabled. You submit a large job A to the cluster...
  • tianbaochao
  • tianbaochao
  • 2016年06月20日 14:41
  • 974

ccah-500 第32题 a new user on the cluster can submit jobs into their own queue application submission

32.Your cluster is running MapReduce version 2 (MRv2) on YARN. Your ResourceManager is configured to...
  • tianbaochao
  • tianbaochao
  • 2016年06月15日 11:02
  • 519

ccah-500 第9题 How would you tune your io.sort.mb value to achieve maximum memory to disk I/O ratio?

9.You observed that the number of spilled records from Map tasks far exceeds the number of map out...
  • tianbaochao
  • tianbaochao
  • 2016年06月01日 17:40
  • 745

ccah-500 第27题 where does the Mapper place the intermediate data of each Map Task

27.During the execution of a MapReduce v2 (MRv2) job on YARN, where does the Mapper place the interm...
  • tianbaochao
  • tianbaochao
  • 2016年06月14日 15:59
  • 413

ccah-500 第45题 You want to minimize the chance of data loss in your cluster. What should you do

45.You have A 20 node Hadoop cluster, with 18 slave nodes and 2 master nodes running HDFS High Avail...
  • tianbaochao
  • tianbaochao
  • 2016年06月17日 10:55
  • 1037

ccah-500 第47题 What is the purpose of ZooKeeper in such a configuration

47.You decide to create a cluster which runs HDFS in High Availability mode with automatic failover ...
  • tianbaochao
  • tianbaochao
  • 2016年06月17日 11:15
  • 936

ccah-500 第40题 maintain your MRv1 TaskTracker slot capacities when you migrate. What should you do

40.You are migrating a cluster from MApReduce version 1 (MRv1) to MapReduce version 2(MRv2) on YARN....
  • tianbaochao
  • tianbaochao
  • 2016年06月15日 14:42
  • 707
您举报文章:ccah-500 第5题 How will the Fair Scheduler handle these two jobs?