配置hadoop 使用fair sc…

配置步骤为

1. 将$HADOOP_HOME/contrib/fairscheduler/hadoop-fairscheduler-0.20.2-cdh3u5.jar拷贝到$HADOOP_HOME/lib文件夹中

2. 修改$HADOOP_HOME/conf/mapred-site.xml配置文件

 

 

1. <</SPAN>property>
2. <</SPAN>name>mapred.jobtracker.taskScheduler</</SPAN>name>
3. <</SPAN>value>org.apache.hadoop.mapred.FairScheduler</</SPAN>value>
4. </</SPAN>property>
5. <</SPAN>property>
6. <</SPAN>name>mapred.fairscheduler.allocation.file</</SPAN>name>
7. <</SPAN>value>/home/hadoop/hadoop-0.20.2-cdh3u5/conf/fair-scheduler.xml</</SPAN>value>
8. </</SPAN>property>
9. <</SPAN>property>
10. <</SPAN>name>mapred.fairscheduler.preemption</</SPAN>name>
11. <</SPAN>value>true</</SPAN>value>
12. </</SPAN>property>
13. <</SPAN>property>
14. <</SPAN>name>mapred.fairscheduler.assignmultiple</</SPAN>name>
15. <</SPAN>value>true</</SPAN>value>
16. </</SPAN>property>
17. <</SPAN>property>
18. <</SPAN>name>mapred.fairscheduler.poolnameproperty</</SPAN>name>
19. <</SPAN>value>mapred.queue.name</</SPAN>value>
20. <</SPAN>description>job.set("mapred.queue.name",pool); // pool is set to either 'high' or 'low' </</SPAN>description>
21. </</SPAN>property>
22. <</SPAN>property>
23. <</SPAN>name>mapred.fairscheduler.preemption.only.log</</SPAN>name>
24. <</SPAN>value>true</</SPAN>value>
25. </</SPAN>property>
26. <</SPAN>property>
27. <</SPAN>name>mapred.fairscheduler.preemption.interval</</SPAN>name>
28. <</SPAN>value>15000</</SPAN>value>
29. </</SPAN>property>
30. <</SPAN>property>
31. <</SPAN>name>mapred.queue.names</</SPAN>name>
32. <</SPAN>value>default,hadoop,hive</</SPAN>value>
33. </</SPAN>property>

 
    mapred.jobtracker.taskScheduler
    org.apache.hadoop.mapred.FairScheduler
  
  
      mapred.fairscheduler.allocation.file
      /home/hadoop/hadoop-0.20.2-cdh3u5/conf/fair-scheduler.xml
   

   
     mapred.fairscheduler.preemption
     true
   

   
     mapred.fairscheduler.assignmultiple
     true
   

   
     mapred.fairscheduler.poolnameproperty
     mapred.queue.name
     job.set("mapred.queue.name",pool); // pool is set to either 'high' or 'low' 
   

   
     mapred.fairscheduler.preemption.only.log
     true
   

   
     mapred.fairscheduler.preemption.interval
     15000
   
   
   
     mapred.queue.names
     default,hadoop,hive
   

3. 在$HADOOP_HOME/conf/新建配置文件fair-scheduler.xml