为服务器集群安装Hadoop On Demand

需要为在沙河校区的56台服务器(x101~x156)安装HOD，torque是一个师姐装的，我在一个师兄的带领下，完成了下面的工作，

1、为56台服务器实现两两之间无密码登录

wumimadenglu.exp

#!/usr/bin/expect -f
set password 123456
for {set i 1} {$i<57} {incr i} {
    if {$i<10} {
    set ip x10$i
    puts "it is the first if : $ip"
    } else {
    set ip x1$i
    puts "it is the first else : $ip"
    }
    
    spawn ssh $ip
    expect "*password:*"
    send "$password\r"
    expect eof

    expect "#"
    send "ssh-keygen -t rsa\r"
    expect "*save the key*"
    send "\r"
    expect "*Overwrite (y/n)?*"
    send "y\r"
    expect "*Enter passphrase*"
    send "\r"
    expect "*Enter same passphrase again*"
    send "\r"
    expect eof

    for {set j 1} {$j<57} {incr j} {
       if {$j<10} {
          set target x10$j
          puts "it is the second if : $ip to $target"
       } else {
          set target x1$j
          puts "it is the second else : $ip to $target"
       }
       
       expect "#" 
       send "ssh-copy-id -i ~/.ssh/id_rsa.pub root@$target\r"
       expect "*password:*"
       send "$password\r"
       expect eof
    }
}
close   

ssh-copy-id -i ~/.ssh/id_rsa.pub root@$target 的功能是将本机id_rsa.pub追加到目标服务器~/.ssh/authorized_keys文件末尾

功能等同于：scp ~/.ssh/id_rsa.pub root@$target:~/.ssh/tmp  ;  然后到目标服务器下执行cat  ~/.ssh/tmp  >>  ~/.ssh/authorized_keys

2、为56台服务器安装python

我下载的是Python-2.5.1.tgz，将此安装包放到x101特定目录下，并执行：

　　# tar zxvf Python-2.5.1.tgz

　　# cd Python-2.5.1

　　# ./configure

　　# make

　　# make install

copypython.sh

for((i=2;i<57;i++))
do
  if [ $i -lt 10 ]
  then
     scp /root/Python-2.5.1.tgz root@x10$i:/root/
     echo "it is the first if: x10$i"
  else
     scp /root/Python-2.5.1.tgz root@x1$i:/root/
     echo "it is the first else: x1$i"
  fi
done

jieyapython.exp

#!/usr/bin/expect -f
for {set i 2} {$i<57} {incr i} {
    if {$i<10} {
    set ip x10$i
    puts "it is the first if : $ip"
    } else {
    set ip x1$i
    puts "it is the first else : $ip"
    }   
    spawn ssh $ip
    expect "#"
    send "tar zxvf Python-2.5.1.tgz\r"
    expect "#"
    send "cd Python-2.5.1\r"
    expect "#"
    send "./configure\r"
    expect "#"
    send "make\r"
    expect "#"
    send "make install\r"
    expect "#"
    send "exit\r"
}

3、为56台服务器安装jdk

我下载的是jdk-6u13-linux-i586.bin，将这个文件拷贝到x101特定目录下后，执行

#  chmod +x jdk-6u13-linux-i586.bin
#  ./jdk-6u13-linux-i586.bin

copyjdkfiletoall.sh

for((i=2;i<57;i++))
do
  if [ $i -lt 10 ]
  then
     scp -r /root/jdk1.6.0_13 root@x10$i:/root/
     echo "it is the first if: x10$i"
  else
     scp -r /root/jdk1.6.0_13 root@x1$i:/root/
     echo "it is the first else: x1$i"
  fi
done

4、为56台服务器安装hadoop

我下载的是hadoop-0.20.2.tar.gz，将此安装包放到x101特定目录下，并执行：

#  tar zxvf hadoop-0.20.0.tar.gz

编辑hadoop-0.20.2/conf/masters文件，加上x101 ;  编辑hadoop-0.20.2/conf/slaves文件，加上x102~x156

编辑hadoop-0.20.2/conf/hadoop-env.sh文件， 加上JAVA_HOME路径

编辑hadoop-0.20.2/conf/core-site.xml文件：

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>
        <property>  
                <name>fs.default.name</name>  
                <value>hdfs://x101:9000</value>  
        </property>  
        <property>  
                <name>hadoop.tmp.dir</name>  
                <value>/root/hadoop/tmp</value>  
        </property> 
</configuration>

编辑hadoop-0.20.2/conf/hdfs-site.xml文件：

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>
   <property>  
       <name>dfs.replication</name>  
       <value>2</value>  
   </property>  
  
   <property>  
       <name>dfs.name.dir</name>  
       <value>/root/hadoop/hdfs/name</value>  
   </property>  
  
   <property>  
       <name>dfs.data.dir</name>  
       <value>/root/hadoop/hdfs/data</value>  
   </property>  
</configuration>

编辑hadoop-0.20.2/conf/mapred-site.xml文件，：

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>
        <property>  
                <name>mapred.job.tracker</name>  
                <value>x101:9001</value>  
        </property>  
        <property>  
                <name>mapred.job.reuse.jvm.num.tasks</name>  
                <value>10</value>  
        </property>  

</configuration>

编辑hod/conf/hodrc（修改环境变量）：

[hod]
stream                          = True
java-home                       = /root/jdk1.6.0_13
cluster                         = clus
cluster-factor                  = 1.8
xrs-port-range                  = 32768-65536
debug                           = 3
allocate-wait-time              = 3600
temp-dir                        = /tmp/hod

[ringmaster]
register                        = True
stream                          = False
temp-dir                        = /tmp/hod
http-port-range                 = 8000-9000
work-dirs                       = /tmp/hod/1,/tmp/hod/2
xrs-port-range                  = 32768-65536
debug                           = 3

[hodring]
stream                          = False
temp-dir                        = /tmp/hod
register                        = True
java-home                       = /root/jdk1.6.0_13
http-port-range                 = 8000-9000
xrs-port-range                  = 32768-65536
debug                           = 3

[resource_manager]
queue                           = batch
batch-home                      = /var/spool/torque
id                              = torque
#env-vars                       = /root/Python-2.5.1/python

[gridservice-mapred]
external                        = False
pkgs                            = /root/hadoop/hadoop-0.20.2
tracker_port                    = 8030
info_port                       = 50080

[gridservice-hdfs]
external                        = False
pkgs                            = /root/hadoop/hadoop-0.20.2
fs_port                         = 8020
info_port                       = 50070

copyhadoopfiletoall.sh

for((i=2;i<57;i++))
do
  if [ $i -lt 10 ]
  then
     echo "it is the first if: x10$i begin!"
     scp -r /root/hadoop/hadoop-0.20.2 root@x10$i:/root/hadoop/
     echo "it is the first if: x10$i end!"
  else
     echo "it is the first else: x1$i begin!"
     scp -r /root/hadoop/hadoop-0.20.2 root@x1$i:/root/hadoop/
     echo "it is the first else: x1$i end!"
  fi
  sleep 3
done

5、为56台服务器配置环境变量

environmentValueSet.sh

export JAVA_HOME=/usr/local/jdk1.6.0_13
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/lib:$CLASSPATH
export HADOOP_HOME=/usr/local/hadoop/hadoop-0.20.2
export PATH=$HADOOP_HOME/bin:$PATH
export CLUSTER_NAME=clus
export RM_QUEUE=batch
export RM_HOME=/var/spool/torque
export PYTHON_HOME=/usr/local/Python-2.5.1
export PATH=$PYTHON_HOME:$PATH
export HOD_PYTHON_HOME=/usr/local/Python-2.5.1/python
export HOD_CONF_DIR=/usr/local/hadoop/hadoop-0.20.2/contrib/hod/conf
export HOD_HOME=/usr/local/hadoop/hadoop-0.20.2/contrib/hod
export PATH=$HOD_HOME/bin:$PATH

copyEnvironmentValuetoAll.sh

for((i=1;i<57;i++))
do
  if [ $i -lt 10 ]
  then
     scp /root/jinchao/environmentValueSet.sh root@x10$i:/root/
     echo "it is the first if: x10$i"
  else
     scp /root/jinchao/environmentValueSet.sh root@x1$i:/root/
     echo "it is the first else: x1$i"
  fi
done

addEnvironmentValue.exp

#!/usr/bin/expect -f
for {set i 1} {$i<57} {incr i} {
    if {$i<10} {
    set ip x10$i
    puts "it is the first if : $ip"
    } else {
    set ip x1$i
    puts "it is the first else : $ip"
    }

    spawn ssh $ip
    expect "#"
    send "cat /root/environmentValueSet.sh >> /etc/profile\r"
    expect "#"
    send "exit\r"
}
close

此外，如果x101的环境变量已经设置好了的话，可以更直接：

copyprofiletoall.sh

for((i=2;i<57;i++))
do
  if [ $i -lt 10 ]
  then
     echo "it is the first if: x10$i begin!"
     scp /etc/profile root@x10$i:/etc/
     echo "it is the first if: x10$i end!"
  else
     echo "it is the first else: x1$i begin!"
     scp /etc/profile root@x1$i:/etc/
     echo "it is the first else: x1$i end!"
  fi
done