Hadoop集群升级手记

1 到站点http://archive.cloudera.com/redhat/cdh/3u4/RPMS/noarch/下载hadoop、hive等rpm包
2 分发新软件包到个datanode
   scp -r hadoop-update-20120705 ip.$I:~/

3 hadoop dfsadmin -upgradeProgress status
  在此次更新之前，之前更新是否完成

4 stop-mapred.sh

  /etc/init.d/hadoop-0.20-jobtracker stop

  /etc/init.d/hadoop-0.20-tasktracker stop

5 hadoop fsck / -files -blocks -locations > dfs-v-old-fsck-1.log
  完成文件系统检查
  hadoop dfs -lsr / > dfs-v-old-lsr-1.log
  在本地保存完整的namespace
6 备份集群配置
7 stop-dfs.sh
  /etc/init.d/hadoop-0.20-namenode stop
  /etc/init.d/hadoop-0.20-datanode stop
8 安装新的hadoop版本
  for I　in {157..196};do
      nohup ssh ip.$i "cd hadoop-update-20120705;rpm -ih hadoop-update-20110823/hadoop-0.20-0.20.2+923.97-1.noarch.rpm hadoop-update-20110823/hadoop-0.20-datanode-0.20.2+923.97-1.noarch.rpm hadoop-update-20110823/hadoop-0.20-tasktracker-0.20.2+923.97-1.noarch.rpm "
  done
9 hadoop-daemon.sh start namenode -upgrade
   开始升级，在此过程中hadoop dfsadmin -upgradeProgress status查看状态
10 hadoop dfs -lsr / > dfs-v-new-lsr-0.log
   hadoop fsck / -files -blocks -locations > dfs-v-new-fsck-1.log
   与之前的文件比对
11 start-mapred.sh
   
12 hadoop dfsadmin -finalizeUpgrade
   结束升级
13 失败的华
   hadoop namenode -rollback