需求: 升级CDH5.14.0集群的hive组件,由1.1.0升级为1.2.1,保证数据不丢失;
- 下载hive-1.2.1的安装包;
http://archive.apache.org/dist/hive/hive-1.2.1/apache-hive-1.2.1-bin.tar.gz
-
将安装包拷贝到集群所有节点;
-
创建hive-1.2.1的依赖路径(所有节点);
midir/opt/cloudera/parcels/CDH/lib/hive/lib121
-
解压hive-1.2.1的压缩包,并将解压后hive/lib下所有文件拷贝到lib121目录(所有节点);
-
Coudera 界面停止Hive服务,如果安装了Hue,Oozie,Impala先停掉,再停掉Hive;
-
修改CDH中hive启动文件中的hive-lib属性(所有节点);
vim /opt/cloudera/parcels/CDH/lib/hive/bin/hive
# 修改HIVE_LIB变量
HIVE_LIB=${HIVE_HOME}/lib121
- 更新hadoop上jline jar包,并删除老的jline jar包(所有节点)
ln -s /opt/cloudera/parcels/CDH/lib/hive/lib121/jline-2.12.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/lib/jline-2.12.jar
mv /opt/cloudera/parcels/CDH/lib/hadoop-yarn/lib/jline-2.11.jar /opt/cloudera/parcels/CDH/lib/hadoop-yarn/lib/jline-2.11.jar.bak
- 备份Mysql中hive元数据库;
mysqldump -uroot -p metastore > metastore01.sql
- 升级hive元数据库
cd /opt/softwares/hive-1.2.1/scripts/metastore/upgrade/mysql/
mysql -uroot -p
......
mysql> use metastore;
mysql> source upgrade-0.13.0-to-0.14.0.mysql.sql;
+--------------------------------------------------+
| |
+--------------------------------------------------+
| Upgrading MetaStore schema from 0.13.0 to 0.14.0 |
+--------------------------------------------------+
1 row in set, 1 warning (0.00 sec)
Query OK, 0 rows affected, 1 warning (0.01 sec)
ERROR 1061 (42000): Duplicate key name 'PCS_STATS_IDX'
Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1 Warnings: 0
+-----------------------------------------------------------+
| |
+-----------------------------------------------------------+
| Finished upgrading MetaStore schema from 0.13.0 to 0.14.0 |
+-----------------------------------------------------------+
1 row in set, 1 warning (0.00 sec)
mysql> source upgrade-0.14.0-to-1.1.0.mysql.sql;
+-------------------------------------------------+
| |
+-------------------------------------------------+
| Upgrading MetaStore schema from 0.14.0 to 1.1.0 |
+-------------------------------------------------+
1 row in set, 1 warning (0.00 sec)
Query OK, 0 rows affected, 1 warning (0.00 sec)
Query OK, 0 rows affected, 1 warning (0.00 sec)
Query OK, 1 row affected (0.01 sec)
Rows matched: 1 Changed: 1 Warnings: 0
+----------------------------------------------------------+
| |
+----------------------------------------------------------+
| Finished upgrading MetaStore schema from 0.14.0 to 1.1.0 |
+----------------------------------------------------------+
1 row in set, 1 warning (0.00 sec)
mysql> source upgrade-1.1.0-to-1.2.0.mysql.sql;
+------------------------------------------------+
| |
+------------------------------------------------+
| Upgrading MetaStore schema from 1.1.0 to 1.2.0 |
+------------------------------------------------+
1 row in set, 1 warning (0.00 sec)
Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1 Warnings: 0
+---------------------------------------------------------+
| |
+---------------------------------------------------------+
| Finished upgrading MetaStore schema from 1.1.0 to 1.2.0 |
+---------------------------------------------------------+
1 row in set, 1 warning (0.00 sec)
-
在Cloudera管理界面启动hive;
-
查看升级后的版本;
hive --version
- 登录hive shell查看升级前后数据是否一致;