Hadoop 2.3.0-cdh5.0.2升级Hadoop 2.5.0-cdh5.3.1

一、准备升级包

for line in `cat /home/hadoop/platform/hadoop.list|awk '{print $1}'`; do echo $line;ssh $line "/bin/mkdir /home/hadoop/platform";done

for line in `cat /home/hadoop/platform/hadoop.list|awk '{print $1}'`; do echo $line;scp /home/hadoop/platform/hadoop-2.5.0-cdh5.3.3.tar.gz hadoop@$line:/home/hadoop/platform;done

for line in `cat /home/hadoop/platform/hadoop.list|awk '{print $1}'`; do echo $line;ssh $line "/bin/tar -zvxf /home/hadoop/platform/hadoop-2.5.0-cdh5.3.3.tar.gz -C /home/hadoop/platform/";done

二、做好文件备份信息：

 a、备份namenode信息，即etc/hadoop/hdfs-site.xml下的dfs.namenode.name.dir所指向的文件夹
 cp -r /data/tmp/dfs/name .


 b、备份config文件：hadoop-env.sh，yarn-env.sh，core-site.xml，mapred-site.xml，hdfs-site.xml，yarn-site.xml
cp -r $HADOOP_CONF_DIR .

c、文件信息

hadoop fsck / -files -blocks -locations > dfs-v-old-fsck-1.log

hadoop dfs -lsr / > dfs-v-old-lsr-1.log

hadoop dfsadmin -report > dfs-v-old-report-1.log

三、将原来的hadoop备份一下，防止升级失败回滚

for line in `cat /home/hadoop/platform/hadoop.list|awk '{print $1}'`; do echo $line;ssh $line "/bin/cp -r /opt/hadoop /opt/hadoop2.3.0";done

四，删除原来的软连接，新建软连接到新版本的hadoop

注意：删除软连接 /opt/hadoop而不是 /opt/hadoop/,

for line in `cat /home/hadoop/platform/hadoop.list|awk '{print $1}'`; do echo $line;ssh $line "/bin/rm -rf  /opt/hadoop";done

for line in `cat /home/hadoop/platform/hadoop.list|awk '{print $1}'`; do echo $line;ssh $line "/bin/ln -s /data/platform/hadoop-2.5.0-cdh5.3.1 /opt/hadoop";done

五、 创建一个FsImage

hdfs dfsadmin -rollingUpgrade prepare

hdfs dfsadmin -rollingUpgrade query 检查滚动Image的状态。等一会接着重新运行这个命令直到“Proceed with rolling upgrade”消息出现。

  

六、执行升级命令

a、关闭standby nn1:

hadoop-daemon.sh stop namenode

b、hdfs namenode -rollingUpgrade started

c、关闭active nn2:

hadoop-daemon.sh stop namenode

d、hdfs namenode -rollingUpgrade started

e、启动所yostart-all.sh

 

遇到如下异常：

File system image contains an old layout version -55.

An upgrade to version -59 is required.

Please restart NameNode with the "-rollingUpgrade started"

option if a rolling upgrade is already started;

 or restart NameNode with the "-upgrade" option to start a new upgrade.

修改：

/data/platform/hadoopupgrade/name/current/VERSION

layoutVersion=-55

改成layoutVersion=-59

nohup hdfs namenode -rollingUpgrade started &

datanode启动不了

最好执行回滚：

start-dfs.sh -rollback

参考资料：

http://blog.csdn.net/shubingzhuoxue/article/details/50085407

https://www.zybuluo.com/layor/note/161370