Yarn Fair Scheduler配置

• 1、先关闭yarn, stop-yarn.sh
• 2、开启fair机制：在yarn-site.xml中配置：

<property>
    <name>yarn.resourcemanager.scheduler.class</name>
    <value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler</value>
</property>

• 3、在$HADOOP_HOME/etc/hadoop下创建文件fair-scheduler.xml,并且配置：

<?xml version="1.0"?>
<allocations>
<defaultQueueSchedulingPolicy>fair</defaultQueueSchedulingPolicy>
        <queue name="prod">
                <weight>40</weight>
                <schedulingPolicy>fifo</schedulingPolicy>
        </queue>
        <queue name="dev">
                <weight>60</weight>
                <queue name="eng"/>
                <queue name="science"/>
        </queue>
</allocations>

• 4、同步：
  scp fair-scheduler.xml yarn-site.xml hadoop-twq@slave1:~/bigdata/hadoop-2.7.5/etc/hadoop/
scp fair-scheduler.xml yarn-site.xml hadoop-twq@slave2:~/bigdata/hadoop-2.7.5/etc/hadoop/
• 5、启动yarn, start-yarn.sh

• 6、通过WebUI查看配置是否成功
  

• 在DistributedCount.java中MapReduce程序指定队列运行
  job.getConfiguration().set("mapreduce.job.queuename", "eng");
  运行中通过WebUI可以查看运行过程：
  

 

 

 

 

转载于:https://www.cnblogs.com/jichui/p/10444969.html