Hbase集群间数据迁移

背景：

有两个集群（cluster-a,cluster-b），需要将cluster-a集群的两张表(table1,table2)迁移到cluster-b集群。

hbase版本号：1.2.6

迁移方法：

使用snapshot方法

迁移步骤：

1.在cluserA集群开启表的复制
hbase shell>
alter 'table1',{NAME => 'id', REPLICATION_SCOPE => '1'}
alter 'table2',{NAME => 'id', REPLICATION_SCOPE => '1'}

2.在cluster-b集群建立到cluster-a集群的peer
hbase shell>
add_peer '1','cluster-a-zk1.s.test.com:2181,cluster-a-zk2.s.test.com:2181,cluster-a-zk3.s.test.com:2181:/hbase'

3.在cluster-b集群建表
备注：以下方式预建分区，只适用于rowkey没编码的场景下，如果编码的就不要预分区了。
hbase shell>
create 'table1',{NAME => 'cf',DATA_BLOCK_ENCODING => 'DIFF', COMPRESSION => 'ZSTD', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1', METADATA => {'STORAGE_POLICY' => 'HOT'}},SPLITS=>['1','2','3','4','5','6','7','8','9']

create 'table2',{NAME => 'skynet', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'DIFF', COMPRESSION => 'ZSTD', MIN_VERSIONS => '0',
BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '1'}

4.在cluster-a集群建立到cluster-b集群的peer
hbase shell>
add_peer '1','cluster-b-zk1.s.test.com:2181,cluster-b-zk2.s.test.com:2181,cluster-b-zk3.s.test.com:2181:/hbase'

确认表已经开启复制
list_replicated_tables

5.禁止cluster-a集群的peer复制，并创建snapshot
hbase shell>
disable_peer '1'
snapshot 'table1', 'table1_snapshot_yyyy_mm_dd'
snapshot 'table2', 'table2_snapshot_yyyy_mm_dd'

6.在cluster-a集群集群，做export snapshot
nohup hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot table1_snapshot_yyyy_mm_dd -copy-to hdfs://cluster-b-master1.s.test.com:8020/user/hbase -mappers 10 -bandwidth 30 &
nohup hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot table2_snapshot_yyyy_mm_dd -copy-to hdfs://cluster-b-master1.s.test.com:8020/user/hbase -mappers 10 -bandwidth 30 &

7.导入完成之后, 在cluster-b集群确认快照正确导入
hbase shell>
list_snapshots

disable 'table1'
restore_snapshot 'table1_snapshot_yyyy_mm_dd'
enable 'table1'

disable 'table2'
restore_snapshot 'table2_snapshot_yyyy_mm_dd'
enable 'table2'

8.在cluster-a集群查看复制积压情况
hbase shell>
enable_peer '1'
status 'replication','resource'

9.通过WEBUI查看cluster-b集群各表的写入是否正常。