配置多队列的容量调度器

默认Yarn的配置下，容量调度器只有一条Default队列。在capacity-scheduler.xml中可以配置多条队列，并降低default队列资源占比：

<!-- 指定多队列，增加hive队列 -->
<property>
   
<name>yarn.scheduler.capacity.root.queues</name>
<value>default,hive</value>  
<description>
   
The queues at the this level (root is the root queue).   
</description>
</property>

<!-- 降低default队列资源额定容量为40%，默认100% -->

<property>


<name>yarn.scheduler.capacity.root.default.capacity</name>

   
<value>40</value>

</property>

 

<!-- 降低default队列资源最大容量为60%，默认100% -->

<property>

   
<name>yarn.scheduler.capacity.root.default.maximum-capacity</name>

   
<value>60</value>

</property>

同时为新加队列添加必要属性：

<!-- 指定hive队列的资源额定容量 -->

<property>

   
<name>yarn.scheduler.capacity.root.hive.capacity</name>

   
<value>60</value>

</property>

 

<property>

   
<name>yarn.scheduler.capacity.root.hive.user-limit-factor</name>

   
<value>1</value>

</property>

 

<!-- 指定hive队列的资源最大容量 -->

<property>

    <name>yarn.scheduler.capacity.root.hive.maximum-capacity</name>

   
<value>80</value>

</property>

 

<property>

   
<name>yarn.scheduler.capacity.root.hive.state</name>

   
<value>RUNNING</value>

</property>

 

<property>

   
<name>yarn.scheduler.capacity.root.hive.acl_submit_applications</name>

   
<value>*</value>

</property>

 

<property>

   
<name>yarn.scheduler.capacity.root.hive.acl_administer_queue</name>

   
<value>*</value>

</property>

 

<property>

   
<name>yarn.scheduler.capacity.root.hive.acl_application_max_priority</name>

   
<value>*</value>

</property>

 

<property>

   
<name>yarn.scheduler.capacity.root.hive.maximum-application-lifetime</name>

   
<value>-1</value>

</property>

 

<property>

   
<name>yarn.scheduler.capacity.root.hive.default-application-lifetime</name>

   
<value>-1</value>

</property>

在配置完成后，重启Yarn或者执行yarn rmadmin -refreshQueues刷新队列，就可以看到