HBase跨集群表复制

2019独角兽企业重金招聘Python工程师标准>>>

概述

A集群HBase中有个表testTableCopy, 要将其复制到B集群的HBase中。

使用HBase表复制工具./hbase org.apache.hadoop.hbase.mapreduce.CopyTable

root@host:/tstar/hbase/bin-->./hbase org.apache.hadoop.hbase.mapreduce.CopyTable
Usage: CopyTable [general options] [--starttime=X] [--endtime=Y] [--new.name=NEW] [--peer.adr=ADR] <tablename>

Options:
 rs.class     hbase.regionserver.class of the peer cluster
              specify if different from current cluster
 rs.impl      hbase.regionserver.impl of the peer cluster
 startrow     the start row
 stoprow      the stop row
 starttime    beginning of the time range (unixtime in millis)
              without endtime means from starttime to forever
 endtime      end of the time range.  Ignored if no starttime specified.
 versions     number of cell versions to copy
 new.name     new table's name
 peer.adr     Address of the peer cluster given in the format
              hbase.zookeeer.quorum:hbase.zookeeper.client.port:zookeeper.znode.parent
 families     comma-separated list of families to copy
              To copy from cf1 to cf2, give sourceCfName:destCfName. 
              To keep the same name, just give "cfName"
 all.cells    also copy delete markers and deleted cells
 bulkload     Write input into HFiles and bulk load to the destination table

Args:
 tablename    Name of the table to copy

Examples:
 To copy 'TestTable' to a cluster that uses replication for a 1 hour window:
 $ bin/hbase org.apache.hadoop.hbase.mapreduce.CopyTable --starttime=1265875194289 --endtime=1265878794289 --peer.adr=server1,server2,server3:2181:/hbase --families=myOldCf:myNewCf,cf2,cf3 TestTable 
For performance consider the following general options:
-Dhbase.client.scanner.caching=100
-Dmapred.map.tasks.speculative.execution=false

准备

1. 工具必须在A集群HBase master节点上进行；
2. 在A集群HBase master节点上须要配置B集群HBase的相关主机名到hosts中，原则上是zookeeper及HBase相关节点主机名；
3. 在B集群HBase中预先建好复制的目标表，目标表的表名可以和原始的表名，目标表的列族名和来源表的列族名，可以不同；

执行复制

./hbase -Dhbase.root.logger=INFO,RFA -Dhbase.log.dir=/tmp -Dhbase.log.file=copytable.log org.apache.hadoop.hbase.mapreduce.CopyTable --peer.adr=test61,test62,test63:2181:/hbase --families=1 "tstar:testTableCopy"

 

转载于:https://my.oschina.net/shipley/blog/3036387