一、升级MetaStore
1.更新hive源码
git checkout -b release-0.13.0 release-0.13.0
2.编译hive
export MAVEN_OPTS=-Xmx512m
mvn package -Phadoop-1,dist -Dmaven.test.skip=true
3.配置Hive
1.cp hive-default.xml.template hive-site.xml
2.cp hive-log4j.properties.template hive-log4j.properties
3.配置hive-log4j.properties
log4j.appender.EventCounter=org.apache.hadoop.metrics.jvm.EventCounter
(解决log4j:ERROR Could not instantiate class [org.apache.hadoop.hive.shims.HiveEventCounter]. )
4.配置hive-site.xml
javax.jdo.option.ConnectionURL=jdbc:mysql://10.12.9.156/hivemetastore?createDatabaseIfNotExist=true
javax.jdo.option.ConnectionDriverName=com.mysql.jdbc.Driver
javax.jdo.option.ConnectionUserName=root
javax.jdo.option.ConnectionPassword=123456
hive.metastore.warehouse.dir=/user/hive/warehouse
5.配置hive-config.sh
export HADOOP_HEAPSIZE=${HADOOP_HEAPSIZE:-4096}
(默认最大对内存256M)
6.其他
删除lib/hbase/lib/libthrift-0.8.0.jar
(解决java.lang.NoSuchMethodError: org.apache.thrift.EncodingUtils.setBit(BIZ)B)
二、升级DataDir
1.对于lib中的hive jar文件,用高版本替换低版本
2.对于lib中的hcatlog jar文件,用高版本替换低版本
3.配置hive-site.xml
hive.metastore.uris=http://10.12.9.156:9083
hive.metastore.warehouse.dir=/user/hive/warehouse
4.配置hive-log4j.properties
cp hive-log4j.properties.debug hive-log4j.properties
(解决log4j:ERROR Could not instantiate class [org.apache.hadoop.hive.shims.HiveEventCounter])
三、升级数据库
1.停掉所有MetaStore对Mysql的连接
2.备份老数据库
mysqldump -h10.11.210.198 -uwap -ptiantianxiangshang hivemetastore >hivemetastore.sql
mysqldump --skip-add-drop-table --no-data -h10.11.210.198 -uwap -ptiantianxiangshang hivemetastore >hivemetastore_schema.sql
3.更新数据库
cd scripts/metastore/upgrade/mysql/
mysql -hlocalhost -uroot -p123456 -e"use hivemetastore; source upgrade-0.12.0-to-0.13.0.mysql.sql;"
四、启动MetaStore
1.启动MetaStore
hive --service metastore
2.测试DataDir
datadir hive -e "select uuid, query from custom.commonc_pc_pv where logdate=2014050800 limit 20"