背景:
有两个集群(cluster-a,cluster-b),需要将cluster-a集群的两张表(table1,table2)迁移到cluster-b集群。
hbase版本号:1.2.6
迁移方法:
使用snapshot方法
迁移步骤:
1.在cluserA集群开启表的复制
hbase shell>
alter 'table1',{NAME => 'id', REPLICATION_SCOPE => '1'}
alter 'table2',{NAME => 'id', REPLICATION_SCOPE => '1'}
2.在cluster-b集群建立到cluster-a集群的peer
hbase shell>
add_peer '1','cluster-a-zk1.s.test.com:2181,cluster-a-zk2.s.test.com:2181,cluster-a-zk3.s.test.com:2181:/hbase'
3.在cluster-b集群建表
备注:以下方式预建分区,只适用于rowkey没编码的场景下,如果编码的就不要预分区了。
hbase shell>
create 'table1',{NAME => 'cf',DATA_BLOCK_ENCODING => 'DIFF', COMPRESSION => 'ZSTD', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1', METADATA => {'STORAGE_POLICY' => 'HOT'}},SPLITS=>['1','2','3','4','5','6','7','8','9']
create 'table2',{NAME => 'skynet', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'DIFF', COMPRESSION => 'ZSTD', MIN_VERSIONS => '0',
BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}
4.在cluster-a集群建立到cluster-b集群的peer
hbase shell>
add_peer '1','cluster-b-zk1.s.test.com:2181,cluster-b-zk2.s.test.com:2181,cluster-b-zk3.s.test.com:2181:/hbase'
确认表已经开启复制
list_replicated_tables
5.禁止cluster-a集群的peer复制,并创建snapshot
hbase shell>
disable_peer '1'
snapshot 'table1', 'table1_snapshot_yyyy_mm_dd'
snapshot 'table2', 'table2_snapshot_yyyy_mm_dd'
6.在cluster-a集群集群,做export snapshot
nohup hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot table1_snapshot_yyyy_mm_dd -copy-to hdfs://cluster-b-master1.s.test.com:8020/user/hbase -mappers 10 -bandwidth 30 &
nohup hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot table2_snapshot_yyyy_mm_dd -copy-to hdfs://cluster-b-master1.s.test.com:8020/user/hbase -mappers 10 -bandwidth 30 &
7.导入完成之后, 在cluster-b集群确认快照正确导入
hbase shell>
list_snapshots
disable 'table1'
restore_snapshot 'table1_snapshot_yyyy_mm_dd'
enable 'table1'
disable 'table2'
restore_snapshot 'table2_snapshot_yyyy_mm_dd'
enable 'table2'
8.在cluster-a集群查看复制积压情况
hbase shell>
enable_peer '1'
status 'replication','resource'
9.通过WEBUI查看cluster-b集群各表的写入是否正常。