Hbase的Region Compact算法实现分析

Hbase的Region Compact算法属于一种多路归并的外排算法。这种算法的特点是，待排序文件本身是有序的，同时打开这些文件，顺序遍历并对比它们的首条数据，最后合并输出为一个文件，多个文件遍历时的首条数据用内存堆进行内排。

Hbase在实现该算法的过程中重要的是下面这五个类。
1.org.apache.hadoop.hbase.regionserver.Store
2.org.apache.hadoop.hbase.regionserver.StoreScanner
3.org.apache.hadoop.hbase.regionserver.StoreFileScanner
4.org.apache.hadoop.hbase.regionserver.KeyValueHeap
5.org.apache.hadoop.hbase.regionserver.ScanQueryMatcher

这五个类的关系是
1.Store类调用StoreScanner的next方法，并循环输出kv到合并文件；
2.StoreScanner的作用是负责创建并持有多个输入文件的StoreFileScanner，内部遍历这些StoreFileScanner并通过KeyValueHeap来排序这些输入文件的首条记录；
3.StoreFileScanner的作用是遍历单个输入文件，管理并提供单个输入文件的首条记录；
4.KeyValueHeap的作用就是通过堆来排序每个输入文件的首条记录。
5.ScanQueryMatcher的作用是当输入文件的首条记录来的时候，根据一定的策略判断这条记录到底是该输出还是该跳过。

源代码如下：

1.Store.compact(final List<StoreFile> filesToCompact,
final boolean majorCompaction, final long maxId)
throws IOException {

这个方法重点是对StoreScanner的方法的调用，因此比较简单，关键的是下面有注释的几句。

// calculate maximum key count after compaction (for blooms)
int maxKeyCount = 0;
for (StoreFile file : filesToCompact) {
StoreFile.Reader r = file.getReader();
if (r != null) {
// NOTE: getFilterEntries could cause under-sized blooms if the user
// switches bloom type (e.g. from ROW to ROWCOL)
long keyCount = (r.getBloomFilterType() == family.getBloomFilterType())
? r.getFilterEntries() : r.getEntries();
maxKeyCount += keyCount;
if (LOG.isDebugEnabled()) {
LOG.debug("Compacting " + file +
", keycount=" + keyCount +
", bloomtype=" + r.getBloomFilterType().toString() +
", size=" + StringUtils.humanReadableInt(r.length()) );
}
}
}

// For each file, obtain a scanner:
List<StoreFileScanner> scanners = StoreFileScanner
.getScannersForStoreFiles(filesToCompact, false, false);//将filesToCompact里的每个输入文件，都为它们创建一个StoreFileScanner对象，用于之后遍历和管理每个输入文件的首条记录。

// Make the instantiation lazy in case compaction produces no product; i.e.
// where all source cells are expired or deleted.
StoreFile.Writer writer = null;
try {
InternalScanner scanner = null;
try {
Scan scan = new Scan();
scan.setMaxVersions(family.getMaxVersions());
/* include deletes, unless we are doing a major compaction */
scanner = new StoreScanner(this, scan, scanners, !majorCompaction);//用刚才创建好的scanners为成员变量再创建StoreScanner，StoreScanner才是真正排序这些文件的对象。
int bytesWritten = 0;
// since scanner.next() can return 'false' but still be delivering data,
// we have to use a do/while loop.
ArrayList<KeyValue> kvs = new ArrayList<KeyValue>();
while (scanner.next(kvs)) {//StoreScanner将文件的记录在内部排序好后，通过next方法循环输出
if (writer == null && !kvs.isEmpty()) {
writer = createWriterInTmp(maxKeyCount,
this.compactionCompression);
}
if (writer != null) {
// output to writer:
for (KeyValue kv : kvs) {
writer.append(kv);//直接将这些记录循环输出到合并文件就行了。

// check periodically to see if a system stop is requested
if (Store.closeCheckInterval > 0) {
bytesWritten += kv.getLength();
if (bytesWritten > Store.closeCheckInterval) {
bytesWritten = 0;
if (!this.region.areWritesEnabled()) {
writer.close();
fs.delete(writer.getPath(), false);
throw new InterruptedIOException(
"Aborting compaction of store " + this +
" in region " + this.region +
" because user requested stop.");
}
}
}
}
}
kvs.clear();
}
} finally {
if (scanner != null) {
scanner.close();
}
}
} finally {
if (writer != null) {
writer.appendMetadata(maxId, majorCompaction);
writer.close();
}
}
return writer;
}


2。StoreScanner.StoreScanner(Store store, Scan scan, List<? extends KeyValueScanner> scanners,
boolean retainDeletesInOutput)
throws IOException {

这个是StoreScanner的构造函数，主要做了三方面工作，见下面注释

this.store = store;
this.cacheBlocks = false;
this.isGet = false;
matcher = new ScanQueryMatcher(scan, store.getFamily().getName(),
null, store.ttl, store.comparator.getRawComparator(),
store.versionsToReturn(scan.getMaxVersions()), retainDeletesInOutput);//创建ScanQueryMatcher对象，用于以后判断记录是该过滤还是输出

// Seek all scanners to the initial key
for(KeyValueScanner scanner : scanners) {
scanner.seek(matcher.getStartKey());//每个输入文件都初始化好首条记录
}

// Combine all seeked scanners with a heap
heap = new KeyValueHeap(scanners, store.comparator);//创建堆对象，用于之后每个输入文件首条记录的排序

}


3.StoreScanner.next(List<KeyValue> outResult, int limit) throws IOException {
//DebugPrint.println("SS.next");

这个方法封装了处理heap取出的记录值的逻辑，根据matcher对该值的判断来决定这个值是输出还是跳过，主要是下面三个方法，见注释

checkReseek();

// if the heap was left null, then the scanners had previously run out anyways, close and
// return.
if (this.heap == null) {
close();
return false;
}

KeyValue peeked = this.heap.peek();
if (peeked == null) {
close();
return false;
}

// only call setRow if the row changes; avoids confusing the query matcher
// if scanning intra-row
if ((matcher.row == null) || !peeked.matchingRow(matcher.row)) {
matcher.setRow(peeked.getRow());
}

KeyValue kv;
List<KeyValue> results = new ArrayList<KeyValue>();
LOOP: while((kv = this.heap.peek()) != null) {//拿出heap里排好序的值
// kv is no longer immutable due to KeyOnlyFilter! use copy for safety
KeyValue copyKv = new KeyValue(kv.getBuffer(), kv.getOffset(), kv.getLength());
ScanQueryMatcher.MatchCode qcode = matcher.match(copyKv);//判断该值该如何处理，输出或跳过
//DebugPrint.println("SS peek kv = " + kv + " with qcode = " + qcode);
switch(qcode) {//根据判断具体的去处理该值
case INCLUDE:
results.add(copyKv);
this.heap.next();
if (limit > 0 && (results.size() == limit)) {
break LOOP;
}
continue;

case DONE:
// copy jazz
outResult.addAll(results);
return true;

case DONE_SCAN:
close();

// copy jazz
outResult.addAll(results);

return false;

case SEEK_NEXT_ROW:
// This is just a relatively simple end of scan fix, to short-cut end us if there is a
// endKey in the scan.
if (!matcher.moreRowsMayExistAfter(kv)) {
outResult.addAll(results);
return false;
}

reseek(matcher.getKeyForNextRow(kv));
break;

case SEEK_NEXT_COL:
reseek(matcher.getKeyForNextColumn(kv));
break;

case SKIP:
this.heap.next();
break;

case SEEK_NEXT_USING_HINT:
KeyValue nextKV = matcher.getNextKeyHint(kv);
if (nextKV != null) {
reseek(nextKV);
} else {
heap.next();
}
break;

default:
throw new RuntimeException("UNEXPECTED");
}
}

if (!results.isEmpty()) {
// copy jazz
outResult.addAll(results);
return true;
}

// No more keys
close();
return false;
}


4.KeyValueHeap.KeyValueHeap(List<? extends KeyValueScanner> scanners,
KVComparator comparator) {

排序堆的构造方法，原来就是一个PriorityQueue，因为要比较的对象是kv记录，所以将kv的comparator作为参数来生成PriorityQueue的comparator

this.comparator = new KVScannerComparator(comparator);
if (!scanners.isEmpty()) {
this.heap = new PriorityQueue<KeyValueScanner>(scanners.size(),
this.comparator);//堆内部就是一个PriorityQueue
for (KeyValueScanner scanner : scanners) {
if (scanner.peek() != null) {
this.heap.add(scanner);//把要排的文件scanner装入堆，文件自动按首条记录通过comparator排好序
} else {
scanner.close();
}
}
this.current = heap.poll();//堆排序后的首个文件scanner
}
}


5.KeyValueHeap.next() throws IOException {

堆里面最重要的方法其实就是next，不过看这个方法的主要功能不是为了算出nextKeyValue，而主要是为了算出nextScanner，然后需在外部再次调用peek方法来取得nextKeyValue，不是很简练。


if(this.current == null) {
return null;
}
KeyValue kvReturn = this.current.next();
KeyValue kvNext = this.current.peek();
if (kvNext == null) {//如果currentScanner已经读完
this.current.close();
this.current = this.heap.poll();//直接从余下scanner中取出新的nextScanner

} else {//如果currentScanner还有值，把余下的scanner中首值最小的scanner取出来比一比，谁小谁就是新的nextScanner

KeyValueScanner topScanner = this.heap.peek();
if (topScanner == null ||
this.comparator.compare(kvNext, topScanner.peek()) >= 0) {
this.heap.add(this.current);
this.current = this.heap.poll();
}
}
return kvReturn;
}


总结：Hbase的Region Compact算法是在一个RegionServer上执行的，在这个执行的过程中，可能有两个瓶颈，
第一，如果待排序的文件数目过多，有导致内存堆溢出的风险，虽然在内存很大的今天，这种情形应该较难出现，但是如果一个regionserver管理了很多region，而这些region又恰好同时都在执行compact的话这就很难说了；
第二，文件的合并工作是在一台RegionServer上执行的，为何不考虑用mapred的形式通过整个集群来分担这种文件合并的工作，最后regionserver只需把合并好的文件装入region就行。