【需求】
在集群A建表,需要同步到集群B并修复历史分区。表用到的数据源在云上。
【操作过程】先在老集群A中获取指定表的建表语句,把建表语句放到文件中,再把该文件发送到B集群中,最后用hive -f 执行该文件即可在B集群批量建表。
【注意】hive -f时涉及的HDFS操作权限问题
A集群有9个表要同步到B集群
[hive@bigdata-master-1 ~]$ vim tables_ToBeCreated_onOtherCluster
# 9个表是:
[hive@bigdata-master-1 ~]$ cat tables_ToBeCreated_onOtherCluster
tony_db.member_sign_gift
tony_db.member_sign_in
tony_db.member_sign_in_addition
tony_db.member_sign_in_month
tony_db.member_sign_user_day_award
tony_db.member_sign_user_month_award
tony_db.member_sign_day_reward
tony_db.member_sign_month_reward
tony_db.member_sign_recharge_config
[hive@bigdata-master-1 ~]$ cat tables_ToBeCreated_onOtherCluster | while read tb; do hive -e "SHOW CREATE TABLE $tb" | grep -vE "Logging|OK|seconds" >> create_table.sql; echo finished ____ $tb; done
Logging initialized using configuration in file:/etc/hive/2.5.0.0-1245/0/hive-log4