YARN-SLS支持多维资源输入

SLS在较新版本中，增加了对多维资源输入的支持。

但遗憾的是，没有实现在web监控界面中对新建资源的监控。

 

1.在/etc/hadoop下新建resource-types.xml文件：

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
       Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<configuration>
<property>
 <name>yarn.resource-types</name>
 <value>resource1,resource2</value>
</property>

<property>
 <name>yarn.resource-types.resource1.units</name>
 <value>G</value>
</property>

<property>
 <name>yarn.resource-types.resource2.units</name>
 <value>G</value>
</property>

</configuration>

表示设置资源的名字为resource1和resource2.单位都为G

 

 

2.修改sls-runner.xml

为NM添加这两个新建资源的值，也就是固定容量

 <property>
   <name>yarn.sls.nm.resource1</name>
   <value>5</value>
 </property>

 <property>
   <name>yarn.sls.nm.resource2</name>
   <value>5</value>
 </property>

 

 

3.修改fair-schedule.xml

把新建的两个资源也添加进调度资源中：

  <userMaxAppsDefault>1000</userMaxAppsDefault>
  <queue name="sls_queue_1">
    <minResources>1024 mb, 1 vcores, 1 resource1</minResources>
    <schedulingMode>fair</schedulingMode>
    <weight>0.25</weight>

 

 

4.修改sls-jobs.json

也就是修改输入的task的需求。添加对这两个资源的需求：

  "job.tasks" : [ {
    "container.host" : "/default-rack/a2118.smile.com",
    "container.start.ms" : 6664,
    "container.end.ms" : 23707,
    "container.priority" : 20,
    "container.memory-mb":1024,
    "container.vcores":1,
    "container.resource1": 1,
    "container.resource2": 1,
    "container.type" : "map"

 

这里要注意，不能对job字段添加这两个资源信息。只能对job中具体的每个task字段添加。否则报错。