hadoop集群配置机架感知

自己搭建的hadoop集群一般默认不具备机架感知能力，为了提高hadoop集群的网络性能，我们常常需要配置机架感知。下面是以下步骤：

一、修改配置文件core-site.xml，添加配置属性：

 

<property>
<name>topology.script.file.name</name>
<value>/home/r203/hadoop-0.20.2/rackaware.py</value>
</property>

　　

二、编写自己的脚本rackaware.py：

 

#!/usr/bin/python
#-*-coding:UTF-8 -*-
import sys

rack = {
        "14a":"rack1","14b":"rack1","14c":"rack1","14d":"rack1","14e":"rack1","14f":"rack1","14g":"rack1","14h":"rack1",
        "14i":"rack1","14j":"rack1","14k":"rack1","14l":"rack1","14m":"rack1","14n":"rack1","14o":"rack1","14p":"rack1",
        "14q":"rack1","14r":"rack1","14s":"rack1","14t":"rack1","14u":"rack1","14v":"rack1","14w":"rack1","14x":"rack1",
        "15a":"rack2","15b":"rack2","15c":"rack2","15d":"rack2","15e":"rack2","15f":"rack2","15g":"rack2","15h":"rack2",
        "15i":"rack2","15j":"rack2","15k":"rack2","15l":"rack2","15m":"rack2","15n":"rack2","15o":"rack2","15p":"rack2",
        "15q":"rack2","15r":"rack2","15s":"rack2","15t":"rack2","15u":"rack2","15v":"rack2","15w":"rack2","15x":"rack2",
        "16a":"rack3","16b":"rack3","16c":"rack3","16d":"rack3","16e":"rack3","16f":"rack3","16g":"rack3","16h":"rack3",
        "16i":"rack3","16j":"rack3","16k":"rack3","16l":"rack3","16m":"rack3","16n":"rack3","16o":"rack3","16p":"rack3",
        "16q":"rack3","16r":"rack3","16s":"rack3","16t":"rack3","16u":"rack3","16v":"rack3","16w":"rack3","16x":"rack3",
       }

if __name__=="__main__":
    print "/" + rack.get(sys.argv[1],"rack0")

　　

注：脚本中可以写主机名也可以写ip地址。

 

 

三、重启namenode。