HBase Filters, Counters & Coprocessors

- Scan, setCaching(rows), setBatch(cells)

- Filter -> FilterBase. setFilter(filter) method on Get and Scan

- CompareFilter, operator + comparator , matched data is kept

CompareFilter(CompareOp valueCompareOp, WritableByteArrayComparable valueComparator)

• operator, LESS, LESS_OR_EQUAL, EQUAL, NOT_EQUAL, GREATER_OR_EQUAL, GREATER, NO_OP
• comparators, extends from WritableByteArrayComparable. BinaryComparator, BinaryPrefixComparator, RegexStringComparator, SubstringComparator,..
• Filters, RowFilter, FamilyFilter, QualifierFilter, ValueFilter, DependentColumnFilter(columns retrieved having same timestamp as referenced column)

Filter filter1 = new RowFilter(CompareFilter.CompareOp.LESS_OR_EQUAL,
new BinaryComparator(Bytes.toBytes("row-22")));

Filter filter = new ValueFilter(CompareFilter.CompareOp.EQUAL, new SubstringComparator(".4"));
Scan scan = new Scan();
scan.setFilter(filter);
ResultScanner scanner = table.getScanner(scan);
for (Result result : scanner) {
    for (KeyValue kv : result.raw()) {
        System.out.println("KV: " + kv + ", Value: " +
            Bytes.toString(kv.getValue()));
    }
}
scanner.close();

- Dedicated filters ,  extends from FilterBase. It filters rows. suitable for scan, not get

• SingleColumnValueFilter, 

SingleColumnValueFilter(byte[] family, byte[] qualifier, CompareOp compareOp, byte[] value)
SingleColumnValueFilter(byte[] family, byte[] qualifier, CompareOp compareOp, WritableByteArrayComparable comparator)

• PrefixFilter
  
• PageFilter

Filter filter = new PageFilter(15);
int totalRows = 0;
byte[] lastRow = null;
while (true) {
    Scan scan = new Scan();
    scan.setFilter(filter);
    if (lastRow != null) {
        byte[] startRow = Bytes.add(lastRow, POSTFIX);//start row
        System.out.println("start row: " +  Bytes.toStringBinary(startRow));
        scan.setStartRow(startRow);
    }
    ResultScanner scanner = table.getScanner(scan);
    int localRows = 0;
    Result result;
    while ((result = scanner.next()) != null) {
        System.out.println(localRows++ + ": " + result);
        totalRows++;
        lastRow = result.getRow();
    }
    scanner.close();
    if (localRows == 0) break;
}
System.out.println("total rows: " + totalRows);

• KeyOnlyFilter, FirstKeyOnlyFilter(first column get), InclusiveStopFilter, TimestampsFilter (TimestampsFilter(List<Long> timestamps)), ColumnCountGetFilter (suitable for get), ColumnPaginationFilter, ColumnPrefixFilter

- Decorating Filters, SkipFilter (skip row if filtered),  WhileMatchFilter (skip later scan if filtered)

- FilterList, list of filters (operators, pass all or pass one)

- Counters, instead of check-and-modify

- Coprocessors, 

• Observer (like a trigger)

1. RegionObserver
2. MasterObserver
3. WALObserver

• Endpoint (like a stored procedure)

public class EndpointExample {
public static void main(String[] args) throws IOException {
Configuration conf = HBaseConfiguration.create();
HTable table = new HTable(conf, "testtable");
try {
Map<byte[], Long> results = table.coprocessorExec(
RowCountProtocol.class,null, null,
new Batch.Call<RowCountProtocol, Long>() {
@Override
public Long call(RowCountProtocol counter) throws IOException {
return counter.getRowCount();
}
});
long total = 0;
for (Map.Entry<byte[], Long> entry : results.entrySet()) {
total += entry.getValue().longValue();
System.out.println("Region: " + Bytes.toString(entry.getKey()) +
", Count: " + entry.getValue());
}
System.out.println("Total Count: " + total);
} catch (Throwable throwable) {
throwable.printStackTrace();
}
}
}