在此先道歉,先前值做了标题,没来得及贴干货在博客,对不住那些提前进来的人,发现访问量较多立马更新博客,在此奉上。
github 地址请查收,使用说明见readme.
https://github.com/matiji66/hbase-value-comparator
这个自定义的Comparator也是看了好多资料,此处是基于cdh-hbase1.2.0 分享自定义hbase filter全过程(此处的Comparator是针对SingleColumnFilter进行测试的,其他的过滤器并没具体测试)。
1.制作 CustomComparatorProtos.proto文件
先编辑一个文件命名为CustomComparatorProtos.proto
具体内容如下:
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
// This file contains protocol buffers that are used for filters
option java_package = "com.pateo.hbase.defined.comparator";
option java_outer_classname = "MyComparatorProtos";
option java_generic_services = true;
option java_generate_equals_and_hash = true;
option optimize_for = SPEED;
// This file contains protocol buffers that are used for comparators (e.g. in filters)
message CustomNumberComparator {
required bytes value = 1;
required string fieldType = 2;
}
2. 使用protoc编辑器进行编译
在protoc 当前的目录 打开终端进入当前目录执行
protoc --java_out=D:\proto CustomNumberComparator.proto
然后得到MyComparatorProtos.java
3. 创建package和上面的填写额一致(com.pateo.hbase.defined.comparator),将MyComparatorProtos.java导入自己定义的package下面,并且在该package下面创建CustomNumberComparator.java继承ByteArrayComparable
//具体代码如下:
package com.pateo.hbase.defined.comparator;
import org.apache.hadoop.hbase.exceptions.DeserializationException;
import org.apache.hadoop.hbase.filter.ByteArrayComparable;
import org.apache.hadoop.hbase.util.Bytes;
import com.google.protobuf.ByteString;
import com.google.protobuf.InvalidProtocolBufferException;
/**
* 自定义比较器:使用方法见 CompareTest
*
* @param : fieldType 传递数据格式的类型,支持的数据类型:double
* @param : data 通过Bytes转换得到的字节数组
* 使用注意事项 : 使用的时候要注意数据类型的匹配问题
*/
public class CustomNumberComparator extends ByteArrayComparable {
/**
* 目前只支持 double类型
*/
private String fieldType;
private byte [] data;
/**
* Constructor
* @param value value
*/
public CustomNumberComparator(byte[] value,String fieldType) {
super(value);
this.fieldType = fieldType;
this.fieldType = "double";
this.data = value;
}
@Override //重写该方法
public byte[] toByteArray() {
CustomComparatorProtos.CustomNumberComparator.Builder builder =
CustomComparatorProtos.CustomNumberComparator.newBuilder();
builder.setValue(ByteString.copyFrom(this.data));
builder.setFieldType(this.fieldType);
return builder.build().toByteArray();
}
//定义该方法,用于对象反序列化操作
public static CustomNumberComparator parseFrom(final byte [] bytes) throws DeserializationException {
CustomComparatorProtos.CustomNumberComparator proto = null;
try {
proto = CustomComparatorProtos.CustomNumberComparator.parseFrom(bytes);
} catch (InvalidProtocolBufferException e) {
throw new DeserializationException(e);
}
return new CustomNumberComparator(proto.getValue().toByteArray(),proto.getFieldType());
}
//重写比较方法 里面就可以按照自己的意愿来实现自己的比较器
@Override
public int compareTo(byte[] bytes, int offset, int length) {
if(fieldType.equalsIgnoreCase("int") || fieldType.equalsIgnoreCase("integer")) {
Integer paramValue = byteConvertObj(Integer.class,this.data);
Integer currentValue = byteConvertObj(Integer.class,Bytes.copy(bytes, offset, length));
return paramValue.compareTo(currentValue);
}else if(fieldType.equalsIgnoreCase("long") || fieldType.equalsIgnoreCase("bigint")){
Long paramsValue = byteConvertObj(Long.class,this.data);
Long currentValue = byteConvertObj(Long.class,Bytes.copy(bytes, offset, length));
return paramsValue.compareTo(currentValue);
}else if(fieldType.equalsIgnoreCase("float")){
Float paramsValue = byteConvertObj(Float.class,this.data);
Float currentValue = byteConvertObj(Float.class,Bytes.copy(bytes, offset, length));
return paramsValue.compareTo(currentValue);
}else if(fieldType.equalsIgnoreCase("double")){
Double paramsValue = byteConvertObj(Double.class,this.data);
Double currentValue = byteConvertObj(Double.class,Bytes.copy(bytes, offset, length));
return paramsValue.compareTo(currentValue);
}else if(fieldType.equalsIgnoreCase("short") || fieldType.equalsIgnoreCase("SMALLINT")){
Short paramsValue = byteConvertObj(Short.class,this.data);
Short currentValue = byteConvertObj(Short.class,Bytes.copy(bytes, offset, length));
return paramsValue.compareTo(currentValue);
}
return 1;
}
private <T> T byteConvertObj(Class<T> clazz,byte [] data){
String clazzName = clazz.getSimpleName();
if(clazzName.equalsIgnoreCase("Integer")){
Integer paramValue ;
try {
paramValue = Bytes.toInt(data);
} catch (IllegalArgumentException e) {
paramValue = Integer.valueOf(Bytes.toString(data));
}
return (T)paramValue;
}else if(clazzName.equalsIgnoreCase("Long")){
Long paramValue ;
try {
paramValue = Bytes.toLong(data);
} catch (IllegalArgumentException e) {
paramValue = Long.valueOf(Bytes.toString(data));
}
return (T)paramValue;
}else if(clazzName.equalsIgnoreCase("Float")){
Float paramValue ;
try {
paramValue = Bytes.toFloat(data);
} catch (IllegalArgumentException e) {
paramValue = Float.valueOf(Bytes.toString(data));
}
return (T)paramValue;
}else if(clazzName.equalsIgnoreCase("Double")){
Double paramValue;
try {
paramValue = Bytes.toDouble(data);
} catch (IllegalArgumentException e) {
paramValue = Double.valueOf(Bytes.toString(data));
}
return (T)paramValue;
}else if(clazzName.equalsIgnoreCase("Short")){
Short paramValue;
try {
paramValue = Bytes.toShort(data);
} catch (IllegalArgumentException e) {
paramValue = Short.valueOf(Bytes.toString(data));
}
return (T)paramValue;
}
return (T) Bytes.toString(data);
}
}
4.选中这两个java文件右击export 输出为jar文件
上传到hbase的lib目录下面,重启hbase,在hbase shell中引入以下 java包
import org.apache.hadoop.hbase.filter.CompareFilter
import org.apache.hadoop.hbase.filter.SingleColumnValueFilter
import com.pateo.hbase.defined.comparator.CustomNumberComparator
scan 'test:user', {COLUMNS => 'basic:age', FILTER => SingleColumnValueFilter.new(Bytes.toBytes('basic'), Bytes.toBytes('age'), CompareFilter::CompareOp.valueOf('GREATER'),CustomNumberComparator.new(Bytes.toBytes(2.0),"double"))}
注意使用说明:
1.
服务器是64位的导致,然后使用的是32位的2.5.0 protoc编译器,数据位数不一致,而64位和32 位的double都是8个字节,所以目前只支持double类型的数据,传入的数值类型也要为double(如2.0)。
2.
当数据值有空的时候,最好使用filterList 结合其他额过滤器进行使用,不然的话会报错的。
例如:
FilterList filterList = new FilterList();
SingleColumnValueFilter nullFilter = new SingleColumnValueFilter(
Bytes.toBytes("basic"),
Bytes.toBytes("age"),
CompareFilter.CompareOp.NOT_EQUAL,
new BinaryComparator(Bytes.toBytes(""))
);
filterList.addFilter(nullFilter);
SingleColumnValueFilter scvf = new SingleColumnValueFilter(
Bytes.toBytes("basic"),
Bytes.toBytes("age"),
CompareFilter.CompareOp.GREATER,
new CustomNumberComparator(Bytes.toBytes(18.0),"double" )// 18.0 注意是小数才行
// new CustomNumberComparator(Bytes.toBytes(18),"int" ) // 支持性不好
// new CustomNumberComparator(Bytes.toBytes(18.0),"float" ) // 支持性不好
);
filterList.addFilter(scvf);
SingleColumnValueFilter bounds = new SingleColumnValueFilter(
Bytes.toBytes("basic"),
Bytes.toBytes("age"),
CompareFilter.CompareOp.LESS,
new CustomNumberComparator(Bytes.toBytes(80.0),"double" )
// new CustomNumberComparator(Bytes.toBytes(18),"int" ) // 支持性不好
// new CustomNumberComparator(Bytes.toBytes(18.0),"float" ) // 支持性不好
);
filterList.addFilter(bounds);
scan.setFilter(filterList);
================================================================
项目文件已经发到github上分享。
附上其他过滤使用说明:
HBase之过滤器
filter ==> SQL 中的Where
filter的执行流程:
过滤器在客户端创建,然后通过RPC发送到服务器上,由服务器执行
基础过滤器:
比较器:
Comparator
Description
LongComparator
Assumes the given value array is a Java Long number and uses Bytes.toLong() to convert it.
BinaryComparator
Uses Bytes.compareTo() to compare 当前值与阀值
BinaryPrefixComparator
Bytes.compareTo() 进行匹配,但是从左端开始前缀匹配
NullComparator
判断当前值是否为null
BitComparator
Performs a bitwise comparison, providing a BitwiseOp enumeration with AND, OR, and XOR operators.
RegexStringComparator
正则表达式匹配
SubstringComparator
子字符串比对
RowFilter 行键过滤器:
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.*;
import org.apache.hadoop.hbase.filter.*;
import org.apache.hadoop.hbase.util.Bytes;
import java.io.IOException;
/**
* 基于行键上的过滤器
*/
public class FilterInHbase {
public static void main(String[] args) throws IOException{
Configuration configuration = HBaseConfiguration.create();
Connection connection = ConnectionFactory.createConnection(configuration);
//建立user表的连接
Table table =connection.getTable(TableName.valueOf(“user”));
Scan scan=new Scan();
//扫描列族info 列age
scan.addColumn(Bytes.toBytes(“info”),Bytes.toBytes(“age”));
System.out.println("行过滤器");
//比较过滤器
//这儿是指找出行小于或者等于"510824118261011172"的所有行
Filter filter1 = new RowFilter(CompareFilter.CompareOp.LESS_OR_EQUAL, new BinaryComparator(Bytes.toBytes("813782218261011172")));
//添加过滤器到扫描器中
scan.setFilter(filter1);
ResultScanner scanner1 = table.getScanner(scan);
for(Result res:scanner1){
System.out.println(res);
}
scanner1.close();
System.out.println("正则过滤器");
//正则过滤器
//过滤行键以2结束的
Filter filter2 = new RowFilter(CompareFilter.CompareOp.EQUAL,
new RegexStringComparator(".*2$")
);
scan.setFilter(filter2);
ResultScanner scanner2 = table.getScanner(scan);
for (Result res:scanner2){
System.out.println(res);
}
scanner2.close();
//子串过滤器
//过滤行键中包含了"61826"这个字符串
System.out.println("子串过滤器");
Scan scan3=new Scan();
//扫描列族info 列age
scan3.addColumn(Bytes.toBytes("info"),Bytes.toBytes("age"));
Filter filter3=new RowFilter(CompareFilter.CompareOp.EQUAL,
new SubstringComparator("61826")
);
scan3.setFilter(filter3);
ResultScanner scanner3=table.getScanner(scan3);
for(Result res:scanner3){
System.out.println(res);
}
scanner3.close();
table.close();
connection.close();
}
}
/**
Result:
行过滤器 < 813782218261011172
keyvalues={224382618261914241/info:age/1472196211169/Put/vlen=2/seqid=0}
keyvalues={510824118261011172/info:age/1472196213020/Put/vlen=2/seqid=0}
keyvalues={524382618264914241/info:age/1472196193913/Put/vlen=2/seqid=0}
keyvalues={673782618261019142/info:age/1472196211733/Put/vlen=2/seqid=0}
keyvalues={813782218261011172/info:age/1472196212550/Put/vlen=2/seqid=0}
正则过滤器 已2结尾的行键
keyvalues={510824118261011172/info:age/1472196213020/Put/vlen=2/seqid=0}
keyvalues={673782618261019142/info:age/1472196211733/Put/vlen=2/seqid=0}
keyvalues={813782218261011172/info:age/1472196212550/Put/vlen=2/seqid=0}
子串过滤器 包含了”61826”的行键
keyvalues={224382618261914241/info:age/1472196211169/Put/vlen=2/seqid=0}
keyvalues={524382618264914241/info:age/1472196193913/Put/vlen=2/seqid=0}
keyvalues={673782618261019142/info:age/1472196211733/Put/vlen=2/seqid=0}
**/
FamilyFilter列族过滤器:
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.*;
import org.apache.hadoop.hbase.filter.BinaryComparator;
import org.apache.hadoop.hbase.filter.CompareFilter;
import org.apache.hadoop.hbase.filter.FamilyFilter;
import org.apache.hadoop.hbase.filter.Filter;
import org.apache.hadoop.hbase.util.Bytes;
import java.io.IOException;
/**
* 列族过滤器
* 比较列族来返回结果
* 用户可以在列族一级筛选所需数据
*/
public class FamilyFilterInHbase {
public static void main(String[] args) throws IOException {
Configuration configuration = HBaseConfiguration.create();
Connection connection = ConnectionFactory.createConnection(configuration);
//建立表的连接
Table table = connection.getTable(TableName.valueOf(“user”));
//比较过滤器 现在表有2个列族 info ship 当然实际有可能很多哦
//取info < "kiss" <ship < "wings"
Filter filter1 = new FamilyFilter(CompareFilter.CompareOp.GREATER, new BinaryComparator(Bytes.toBytes("kiss")));
Scan scan = new Scan();
scan.setFilter(filter1);
ResultScanner scanner = table.getScanner(scan);
for (Result result : scanner){
System.out.println(result);
}
scanner.close();
Get get1 = new Get(Bytes.toBytes("673782618261019142"));
get1.setFilter(filter1);
Result result1=table.get(get1);
System.out.println("Result of get1(): " + result1);
//添加列族过滤器 info
Filter filter2= new FamilyFilter(CompareFilter.CompareOp.EQUAL,new BinaryComparator(Bytes.toBytes("info")));
//获取一行数据
Get get2 = new Get(Bytes.toBytes("673782618261019142"));
//但是get列族ship 那么==>> Result of get():keyvalues=NONE [本身冲突 所以无数据]
//如果get列族info 那么==>> Result of get2():keyvalues={673782618261019142/...
get2.addFamily(Bytes.toBytes("ship"));
get2.setFilter(filter2);
Result result2 =table.get(get2);
System.out.println("Result of get2():"+result2);
scanner.close();
table.close();
connection.close();
}
}
/**
LESS “kiss”
keyvalues={224382618261914241/info:age/1472196211169/Put/vlen=2/seqid=0, 224382618261914241/info:height/1472196211234/Put/vlen=3/seqid=0, 224382618261914241/info:name/1472196211088/Put/vlen=4/seqid=0, 224382618261914241/info:phone/1472196211427/Put/vlen=11/seqid=0, 224382618261914241/info:weight/1472196211386/Put/vlen=3/seqid=0}
keyvalues={510824118261011172/info:age/1472196213020/Put/vlen=2/seqid=0, 510824118261011172/info:height/1472196213056/Put/vlen=3/seqid=0, 510824118261011172/info:name/1472196212942/Put/vlen=8/seqid=0, 510824118261011172/info:phone/1472196213237/Put/vlen=11/seqid=0, 510824118261011172/info:weight/1472196213169/Put/vlen=3/seqid=0}
keyvalues={524382618264914241/info:age/1472196193913/Put/vlen=2/seqid=0, 524382618264914241/info:height/1472196194783/Put/vlen=3/seqid=0, 524382618264914241/info:name/1472196193255/Put/vlen=8/seqid=0, 524382618264914241/info:phone/1472196195125/Put/vlen=11/seqid=0, 524382618264914241/info:weight/1472196194970/Put/vlen=3/seqid=0}
keyvalues={673782618261019142/info:age/1472196211733/Put/vlen=2/seqid=0, 673782618261019142/info:height/1472196211761/Put/vlen=3/seqid=0, 673782618261019142/info:name/1472196211678/Put/vlen=7/seqid=0, 673782618261019142/info:phone/1472196211956/Put/vlen=11/seqid=0, 673782618261019142/info:weight/1472196211841/Put/vlen=3/seqid=0}
keyvalues={813782218261011172/info:age/1472196212550/Put/vlen=2/seqid=0, 813782218261011172/info:height/1472196212605/Put/vlen=3/seqid=0, 813782218261011172/info:name/1472196212480/Put/vlen=8/seqid=0, 813782218261011172/info:phone/1472196212713/Put/vlen=11/seqid=0, 813782218261011172/info:weight/1472196212651/Put/vlen=3/seqid=0}
GREATER “kiss”
keyvalues={224382618261914241/ship:addr/1472196211487/Put/vlen=7/seqid=0, 224382618261914241/ship:email/1472196211530/Put/vlen=11/seqid=0, 224382618261914241/ship:salary/1472196211594/Put/vlen=4/seqid=0}
keyvalues={510824118261011172/ship:addr/1472196213328/Put/vlen=8/seqid=0, 510824118261011172/ship:email/1472196213422/Put/vlen=12/seqid=0, 510824118261011172/ship:salary/1472196214963/Put/vlen=5/seqid=0}
keyvalues={524382618264914241/ship:addr/1472196195270/Put/vlen=7/seqid=0, 524382618264914241/ship:email/1472196195371/Put/vlen=13/seqid=0, 524382618264914241/ship:salary/1472196195485/Put/vlen=4/seqid=0}
keyvalues={673782618261019142/ship:addr/1472196212059/Put/vlen=8/seqid=0, 673782618261019142/ship:email/1472196212176/Put/vlen=12/seqid=0, 673782618261019142/ship:salary/1472196212284/Put/vlen=4/seqid=0}
keyvalues={813782218261011172/ship:addr/1472196212762/Put/vlen=4/seqid=0, 813782218261011172/ship:email/1472196212802/Put/vlen=12/seqid=0, 813782218261011172/ship:salary/1472196212840/Put/vlen=5/seqid=0}
APPEND(LESS “kiss”)
Result of get(): keyvalues={673782618261019142/info:age/1472196211733/Put/vlen=2/seqid=0, 673782618261019142/info:height/1472196211761/Put/vlen=3/seqid=0, 673782618261019142/info:name/1472196211678/Put/vlen=7/seqid=0, 673782618261019142/info:phone/1472196211956/Put/vlen=11/seqid=0, 673782618261019142/info:weight/1472196211841/Put/vlen=3/seqid=0}
APPEND(GREATER “kiss”)
Result of get1(): keyvalues={673782618261019142/ship:addr/1472196212059/Put/vlen=8/seqid=0, 673782618261019142/ship:email/1472196212176/Put/vlen=12/seqid=0, 673782618261019142/ship:salary/1472196212284/Put/vlen=4/seqid=0}
//filter “info” get “ship”
Result of get():keyvalues=NONE
//filter “info” get “info”
Result of get2():keyvalues={673782618261019142/info:age/1472196211733/Put/vlen=2/seqid=0, 673782618261019142/info:height/1472196211761/Put/vlen=3/seqid=0, 673782618261019142/info:name/1472196211678/Put/vlen=7/seqid=0, 673782618261019142/info:phone/1472196211956/Put/vlen=11/seqid=0, 673782618261019142/info:weight/1472196211841/Put/vlen=3/seqid=0}
**/
ValueFilter值过滤器:
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.Cell;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.*;
import org.apache.hadoop.hbase.filter.*;
import org.apache.hadoop.hbase.util.Bytes;
import java.io.IOException;
/**
* 值过滤器
* 根据值进行筛选 可以联合RegexStringComparator 进行设计
*/
public class FilterOfValue {
public static void main(String[] args) throws IOException {
Configuration configuration = HBaseConfiguration.create();
Connection connection = ConnectionFactory.createConnection(configuration);
//建立表的连接
Table table = connection.getTable(TableName.valueOf(“user”));
//值中包含了177的过滤器
Filter filter = new ValueFilter(CompareFilter.CompareOp.EQUAL,
new SubstringComparator("1771392142")
);
Scan scan = new Scan();
scan.setFilter(filter);
ResultScanner scanner = table.getScanner(scan);
for (Result result : scanner){
for (Cell cell:result.rawCells()){
System.out.println("Cell: "+cell+", Value: "+Bytes.toString(cell.getValueArray(),cell.getValueLength()));
}
}
scanner.close();
Get get1 = new Get(Bytes.toBytes("673782618261019142"));
get1.setFilter(filter);
Result result1=table.get(get1);
for (Cell cell : result1.rawCells()) {
System.out.println("Get1 Cell: " + cell + ", Value: " +
Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
cell.getValueLength()));
}
Get get2 = new Get(Bytes.toBytes("813782218261011172"));
get2.setFilter(filter);
Result result2=table.get(get2);
for (Cell cell : result2.rawCells()) {
System.out.println("Get2 Cell: " + cell + ", Value: " +
Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
cell.getValueLength()));
}
table.close();
connection.close();
}
}
/**
原数据:
673782618261019142 column=info:phone, timestamp=1472196211956, value=17713921424
813782218261011172 column=info:phone, timestamp=1472196212713, value=12713921424
*输出结果:
Cell: 673782618261019142/info:phone/1472196211956/Put/vlen=11/seqid=0, Value: 73782618261019142infophoneVŻt�17713921424
Get1 Cell: 673782618261019142/info:phone/1472196211956/Put/vlen=11/seqid=0, Value: 17713921424
**/
DependentColumnFilter 参考列过滤器
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.Cell;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.*;
import org.apache.hadoop.hbase.filter.*;
import org.apache.hadoop.hbase.util.Bytes;
import java.io.IOException;
/**
* 参考列过滤器
* 根据列名进行筛选
*/
public class FilterOfDependentColumnFilter {
private static Table table=null;
public static Table getTable() {
if(table==null){
try {
Configuration configuration = HBaseConfiguration.create();
Connection connection = ConnectionFactory.createConnection(configuration);
//建立表的连接
return connection.getTable(TableName.valueOf(“user”));
}catch (IOException e){
return table;
}
}
return table;
}
public static void filter(boolean drop,CompareFilter.CompareOp oper,ByteArrayComparable comparable) throws IOException {
Filter filter;
if (comparable != null) {
filter = new DependentColumnFilter(Bytes.toBytes("info"), Bytes.toBytes("phone"), drop, oper, comparable);
} else {
filter = new DependentColumnFilter(Bytes.toBytes("info"), Bytes.toBytes("phone"), drop);
}
Scan scan = new Scan();
scan.setFilter(filter);
ResultScanner scanner = getTable().getScanner(scan);
for (Result result : scanner) {
for (Cell cell : result.rawCells()) {
System.out.println("Cell: " + cell + ", Value: " +
Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
cell.getValueLength()));
}
}
scanner.close();
Get get = new Get(Bytes.toBytes("673782618261019142"));
get.setFilter(filter);
Result result = getTable().get(get);
for (Cell cell : result.rawCells()) {
System.out.println("Cell: " + cell + ", Value: " +
Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
cell.getValueLength()));
}
}
public static void main(String[] args) throws IOException {
filter(true, CompareFilter.CompareOp.NO_OP, null);
filter(false, CompareFilter.CompareOp.NO_OP, null);
filter(true, CompareFilter.CompareOp.EQUAL,
new BinaryPrefixComparator(Bytes.toBytes("17713921424")));
filter(false, CompareFilter.CompareOp.EQUAL,
new BinaryPrefixComparator(Bytes.toBytes("17713921424")));
filter(true, CompareFilter.CompareOp.EQUAL,
new RegexStringComparator(".*\\.5"));
filter(false, CompareFilter.CompareOp.EQUAL,
new RegexStringComparator(".*\\.5"));
}
}
/**
**/
PrefixFilter 前缀过滤器:
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.Cell;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.*;
import org.apache.hadoop.hbase.filter.*;
import org.apache.hadoop.hbase.util.Bytes;
import java.io.IOException;
/**
* 前缀过滤器
* 根据行键的前缀进行过滤
*/
public class FilterOfPrefixFilter {
public static void main(String[] args) throws IOException {
Configuration configuration = HBaseConfiguration.create();
Connection connection = ConnectionFactory.createConnection(configuration);
//建立表的连接
Table table = connection.getTable(TableName.valueOf(“user”));
Filter filter = new PrefixFilter(Bytes.toBytes(“510824”));
Scan scan = new Scan();
scan.setFilter(filter);
ResultScanner scanner = table.getScanner(scan);
for (Result result : scanner) {
for (Cell cell : result.rawCells()) {
System.out.println("Cell: " + cell + ", Value: " +
Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
cell.getValueLength()));
} }
scanner.close();
Get get = new Get(Bytes.toBytes("row-5"));
get.setFilter(filter);
Result result = table.get(get);
for (Cell cell : result.rawCells()) {
System.out.println("Cell: " + cell + ", Value: " +
Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
cell.getValueLength()));
}
scanner.close();
table.close();
connection.close();
}
}
/**
Cell: 510824118261011172/info:height/1472196213056/Put/vlen=3/seqid=0, Value: 188
Cell: 510824118261011172/info:name/1472196212942/Put/vlen=8/seqid=0, Value: yangyang
Cell: 510824118261011172/info:phone/1472196213237/Put/vlen=11/seqid=0, Value: 18013921626
Cell: 510824118261011172/info:weight/1472196213169/Put/vlen=3/seqid=0, Value: 138
Cell: 510824118261011172/ship:addr/1472196213328/Put/vlen=8/seqid=0, Value: shanghai
Cell: 510824118261011172/ship:email/1472196213422/Put/vlen=12/seqid=0, Value: 199@sina.com
Cell: 510824118261011172/ship:salary/1472196214963/Put/vlen=5/seqid=0, Value: 50000
**/
PageFilter 分页过滤器
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.Cell;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.*;
import org.apache.hadoop.hbase.filter.Filter;
import org.apache.hadoop.hbase.filter.PageFilter;
import org.apache.hadoop.hbase.filter.PrefixFilter;
import org.apache.hadoop.hbase.util.Bytes;
import java.io.IOException;
/**
* 分页过滤器
*/
public class FilterOfPage {
private static final byte[] POSTFIX = new byte[] { 0x00 };
public static void main(String[] args) throws IOException {
Configuration configuration = HBaseConfiguration.create();
Connection connection = ConnectionFactory.createConnection(configuration);
//建立表的连接
Table table = connection.getTable(TableName.valueOf("user"));
//分页大小
Filter filter = new PageFilter(3);
int totalRows = 0;
byte[] lastRow = null;
while (true) {
Scan scan = new Scan();
scan.setFilter(filter);
if (lastRow != null) {
byte[] startRow = Bytes.add(lastRow, POSTFIX);
System.out.println("start row: " +
Bytes.toStringBinary(startRow));
scan.setStartRow(startRow);
}
ResultScanner scanner = table.getScanner(scan);
int localRows = 0;
Result result;
while ((result = scanner.next()) != null) {
System.out.println(localRows++ + ": " + result);
totalRows++;
lastRow = result.getRow();
}
scanner.close();
if (localRows == 0) break;
}
System.out.println("total rows: " + totalRows);
table.close();
connection.close();
}
}
/**
0: keyvalues={224382618261914241/info:age/1472196211169/Put/vlen=2/seqid=0, 224382618261914241/info:height/1472196211234/Put/vlen=3/seqid=0, 224382618261914241/info:name/1472196211088/Put/vlen=4/seqid=0, 224382618261914241/info:phone/1472196211427/Put/vlen=11/seqid=0, 224382618261914241/info:weight/1472196211386/Put/vlen=3/seqid=0, 224382618261914241/ship:addr/1472196211487/Put/vlen=7/seqid=0, 224382618261914241/ship:email/1472196211530/Put/vlen=11/seqid=0, 224382618261914241/ship:salary/1472196211594/Put/vlen=4/seqid=0}
1: keyvalues={510824118261011172/info:age/1472196213020/Put/vlen=2/seqid=0, 510824118261011172/info:height/1472196213056/Put/vlen=3/seqid=0, 510824118261011172/info:name/1472196212942/Put/vlen=8/seqid=0, 510824118261011172/info:phone/1472196213237/Put/vlen=11/seqid=0, 510824118261011172/info:weight/1472196213169/Put/vlen=3/seqid=0, 510824118261011172/ship:addr/1472196213328/Put/vlen=8/seqid=0, 510824118261011172/ship:email/1472196213422/Put/vlen=12/seqid=0, 510824118261011172/ship:salary/1472196214963/Put/vlen=5/seqid=0}
2: keyvalues={524382618264914241/info:age/1472196193913/Put/vlen=2/seqid=0, 524382618264914241/info:height/1472196194783/Put/vlen=3/seqid=0, 524382618264914241/info:name/1472196193255/Put/vlen=8/seqid=0, 524382618264914241/info:phone/1472196195125/Put/vlen=11/seqid=0, 524382618264914241/info:weight/1472196194970/Put/vlen=3/seqid=0, 524382618264914241/ship:addr/1472196195270/Put/vlen=7/seqid=0, 524382618264914241/ship:email/1472196195371/Put/vlen=13/seqid=0, 524382618264914241/ship:salary/1472196195485/Put/vlen=4/seqid=0}
start row: 524382618264914241\x00
0: keyvalues={673782618261019142/info:age/1472196211733/Put/vlen=2/seqid=0, 673782618261019142/info:height/1472196211761/Put/vlen=3/seqid=0, 673782618261019142/info:name/1472196211678/Put/vlen=7/seqid=0, 673782618261019142/info:phone/1472196211956/Put/vlen=11/seqid=0, 673782618261019142/info:weight/1472196211841/Put/vlen=3/seqid=0, 673782618261019142/ship:addr/1472196212059/Put/vlen=8/seqid=0, 673782618261019142/ship:email/1472196212176/Put/vlen=12/seqid=0, 673782618261019142/ship:salary/1472196212284/Put/vlen=4/seqid=0}
1: keyvalues={813782218261011172/info:age/1472196212550/Put/vlen=2/seqid=0, 813782218261011172/info:height/1472196212605/Put/vlen=3/seqid=0, 813782218261011172/info:name/1472196212480/Put/vlen=8/seqid=0, 813782218261011172/info:phone/1472196212713/Put/vlen=11/seqid=0, 813782218261011172/info:weight/1472196212651/Put/vlen=3/seqid=0, 813782218261011172/ship:addr/1472196212762/Put/vlen=4/seqid=0, 813782218261011172/ship:email/1472196212802/Put/vlen=12/seqid=0, 813782218261011172/ship:salary/1472196212840/Put/vlen=5/seqid=0}
start row: 813782218261011172\x00
2016-08-29 17:17:57,197 INFO [main] client.ConnectionManager$HConnectionImplementation: Closing zookeeper sessionid=0x56c13890bf003a
total rows: 5
**/
KeyOnlyFilter 行键过滤器
FirstKeyOnlyFilter 首次行键过滤器
FirstKeyValueMatchingQualifiersFilter
InclusiveStopFilter 包含结束的过滤器
FuzzyRowFilter 模糊行匹配过滤器
ColumnCountGetFilter 列计数过滤器
可以使用这个过滤器来限制每行最多取回多少列,
当一行的列数到设定的最大值,这个过滤器会停止整个扫描操作。
适合在get方法中使用
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.Cell;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.*;
import org.apache.hadoop.hbase.filter.ColumnCountGetFilter;
import org.apache.hadoop.hbase.util.Bytes;
import java.io.IOException;
/**
* ColumnCountGetFilter 列数过滤器
*/
public class FilterOfColumnCountGetFilter {
public static void main(String args[]) throws IOException{
Configuration configuration = HBaseConfiguration.create();
Connection connection = ConnectionFactory.createConnection(configuration);
Table table = connection.getTable(TableName.valueOf(“user”));
//限制返回的列数
ColumnCountGetFilter columnCountGetFilter = new ColumnCountGetFilter(3);
// column=info:age,column=info:height,column=info:name,column=info:phone,column=info:weight,
Get get = new Get(Bytes.toBytes("224382618261914241"));
get.setFilter(columnCountGetFilter);
Result result = table.get(get);
System.out.println("Result of columnCountGetFilter get: ");
for (Cell cell : result.rawCells()) {
System.out.println("Cell: " + cell + ", Value: " +
Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
cell.getValueLength()));
}
Get get1 = new Get(Bytes.toBytes("224382618261914241"));
Result result1 = table.get(get1);
System.out.println("Result of get: ");
for (Cell cell : result1.rawCells()) {
System.out.println("Cell: " + cell + ", Value: " +
Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
cell.getValueLength()));
}
table.close();
connection.close();
}
}
/**
Result of columnCountGetFilter get:
Cell: 224382618261914241/info:age/1472196211169/Put/vlen=2/seqid=0, Value: 24
Cell: 224382618261914241/info:height/1472196211234/Put/vlen=3/seqid=0, Value: 158
Cell: 224382618261914241/info:name/1472196211088/Put/vlen=4/seqid=0, Value: lisi
Result of get:
Cell: 224382618261914241/info:age/1472196211169/Put/vlen=2/seqid=0, Value: 24
Cell: 224382618261914241/info:height/1472196211234/Put/vlen=3/seqid=0, Value: 158
Cell: 224382618261914241/info:name/1472196211088/Put/vlen=4/seqid=0, Value: lisi
Cell: 224382618261914241/info:phone/1472196211427/Put/vlen=11/seqid=0, Value: 13213921424
Cell: 224382618261914241/info:weight/1472196211386/Put/vlen=3/seqid=0, Value: 128
Cell: 224382618261914241/ship:addr/1472196211487/Put/vlen=7/seqid=0, Value: chengdu
Cell: 224382618261914241/ship:email/1472196211530/Put/vlen=11/seqid=0, Value: qq@sina.com
Cell: 224382618261914241/ship:salary/1472196211594/Put/vlen=4/seqid=0, Value: 5000
**/
ColumnPaginationFilter 列分页过滤
HBase 的列可以很多,所以出现了列分页
该过滤器可以对一行的所有列进行分页
构造函数:
ColumnPaginationFilter(int limit, int offset)
ColumnPaginationFilter(int limit, byte[] columnOffset)
limit 限制取回来列数 offset 偏移位就是开始位置 byte[] 字符串/书签偏移从哪里开始分页。
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.Cell;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.*;
import org.apache.hadoop.hbase.filter.ColumnPaginationFilter;
import org.apache.hadoop.hbase.util.Bytes;
import java.io.IOException;
/**
* ColumnPageFilter 列分页过滤器
*/
public class FilterOColumnPageFilter {
public static void main(String args[]) throws IOException{
Configuration configuration = HBaseConfiguration.create();
Connection connection = ConnectionFactory.createConnection(configuration);
Table table = connection.getTable(TableName.valueOf(“user”));
//限制返回的列数 从第4列开始取 取3列数据
ColumnPaginationFilter columnPaginationFilter = new ColumnPaginationFilter(3,4);
Get get = new Get(Bytes.toBytes(“224382618261914241”));
get.setFilter(columnPaginationFilter);
Result result = table.get(get);
System.out.println(“Result of ColumnPageFilter get: “);
for (Cell cell : result.rawCells()) {
System.out.println(“Cell: ” + cell + “, Value: ” +
Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
cell.getValueLength()));
}
//限制返回的列数 从第name列开始取 取3列数据
ColumnPaginationFilter columnPaginationFilter2 = new ColumnPaginationFilter(3,Bytes.toBytes("name"));
Get get2 = new Get(Bytes.toBytes("224382618261914241"));
get2.setFilter(columnPaginationFilter2);
Result result2 = table.get(get2);
System.out.println("Result of ColumnPageFilter get: ");
for (Cell cell : result2.rawCells()) {
System.out.println("Cell: " + cell + ", Value: " +
Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
cell.getValueLength()));
}
Get get1 = new Get(Bytes.toBytes("224382618261914241"));
Result result1 = table.get(get1);
System.out.println("Result of get: ");
for (Cell cell : result1.rawCells()) {
System.out.println("Cell: " + cell + ", Value: " +
Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
cell.getValueLength()));
}
table.close();
connection.close();
}
}
/**
Result of ColumnPageFilter get:
Cell: 224382618261914241/info:weight/1472196211386/Put/vlen=3/seqid=0, Value: 128
Cell: 224382618261914241/ship:addr/1472196211487/Put/vlen=7/seqid=0, Value: chengdu
Cell: 224382618261914241/ship:email/1472196211530/Put/vlen=11/seqid=0, Value: qq@sina.com
Result of ColumnPageFilter get:
Cell: 224382618261914241/info:name/1472196211088/Put/vlen=4/seqid=0, Value: lisi
Cell: 224382618261914241/info:phone/1472196211427/Put/vlen=11/seqid=0, Value: 13213921424
Cell: 224382618261914241/info:weight/1472196211386/Put/vlen=3/seqid=0, Value: 128
Result of get:
Cell: 224382618261914241/info:age/1472196211169/Put/vlen=2/seqid=0, Value: 24
Cell: 224382618261914241/info:height/1472196211234/Put/vlen=3/seqid=0, Value: 158
Cell: 224382618261914241/info:name/1472196211088/Put/vlen=4/seqid=0, Value: lisi
Cell: 224382618261914241/info:phone/1472196211427/Put/vlen=11/seqid=0, Value: 13213921424
Cell: 224382618261914241/info:weight/1472196211386/Put/vlen=3/seqid=0, Value: 128
Cell: 224382618261914241/ship:addr/1472196211487/Put/vlen=7/seqid=0, Value: chengdu
Cell: 224382618261914241/ship:email/1472196211530/Put/vlen=11/seqid=0, Value: qq@sina.com
Cell: 224382618261914241/ship:salary/1472196211594/Put/vlen=4/seqid=0, Value: 5000
**/
ColumnPrefixFilter 列前缀过虑器
该过滤器通过对列名称进行前缀匹配过滤
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.Cell;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.*;
import org.apache.hadoop.hbase.filter.ColumnPrefixFilter;
import org.apache.hadoop.hbase.filter.Filter;
import org.apache.hadoop.hbase.util.Bytes;
import java.io.IOException;
/**
* ColumnPrefixFilter 列前缀过滤器
*/
public class FilterOfColumnPrefixFilter {
public static void main(String args[]) throws IOException{
Configuration configuration = HBaseConfiguration.create();
Connection connection = ConnectionFactory.createConnection(configuration);
Table table = connection.getTable(TableName.valueOf(“user”));
//取列名已ag开头的
Filter columnPrefixFilter = new ColumnPrefixFilter(Bytes.toBytes(“a”));
Get get = new Get(Bytes.toBytes(“224382618261914241”));
get.setFilter(columnPrefixFilter);
Result result = table.get(get);
System.out.println(“Result of columnPrefixFilter get: “);
for (Cell cell : result.rawCells()) {
System.out.println(“Cell: ” + cell + “, Value: ” +
Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
cell.getValueLength()));
}
Get get1 = new Get(Bytes.toBytes("224382618261914241"));
Result result1 = table.get(get1);
System.out.println("Result of get: ");
for (Cell cell : result1.rawCells()) {
System.out.println("Cell: " + cell + ", Value: " +
Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
cell.getValueLength()));
}
table.close();
connection.close();
}
}
/**
Result of columnPrefixFilter get:
Cell: 224382618261914241/info:age/1472196211169/Put/vlen=2/seqid=0, Value: 24
Cell: 224382618261914241/ship:addr/1472196211487/Put/vlen=7/seqid=0, Value: chengdu
Result of get:
Cell: 224382618261914241/info:age/1472196211169/Put/vlen=2/seqid=0, Value: 24
Cell: 224382618261914241/info:height/1472196211234/Put/vlen=3/seqid=0, Value: 158
Cell: 224382618261914241/info:name/1472196211088/Put/vlen=4/seqid=0, Value: lisi
Cell: 224382618261914241/info:phone/1472196211427/Put/vlen=11/seqid=0, Value: 13213921424
Cell: 224382618261914241/info:weight/1472196211386/Put/vlen=3/seqid=0, Value: 128
Cell: 224382618261914241/ship:addr/1472196211487/Put/vlen=7/seqid=0, Value: chengdu
Cell: 224382618261914241/ship:email/1472196211530/Put/vlen=11/seqid=0, Value: qq@sina.com
Cell: 224382618261914241/ship:salary/1472196211594/Put/vlen=4/seqid=0, Value: 5000
**/
MultipleColumnPrefixFilter 多个列前缀过滤器
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.Cell;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.*;
import org.apache.hadoop.hbase.filter.Filter;
import org.apache.hadoop.hbase.filter.MultipleColumnPrefixFilter;
import org.apache.hadoop.hbase.util.Bytes;
import java.io.IOException;
/**
* MultipleColumnPrefixFilter 多个列前缀过滤器
*/
public class FilterOfMultipleColumnPrefixFilter {
public static void main(String args[]) throws IOException{
Configuration configuration = HBaseConfiguration.create();
Connection connection = ConnectionFactory.createConnection(configuration);
Table table = connection.getTable(TableName.valueOf(“user”));
//取列名已a开头的 还有已h 开头的
Filter filter = new MultipleColumnPrefixFilter(new byte[][] {Bytes.toBytes(“a”),Bytes.toBytes(“h”)});
Get get = new Get(Bytes.toBytes(“224382618261914241”));
get.setFilter(filter);
Result result = table.get(get);
System.out.println(“Result of columnPrefixFilter get: “);
for (Cell cell : result.rawCells()) {
System.out.println(“Cell: ” + cell + “, Value: ” +
Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
cell.getValueLength()));
}
Get get1 = new Get(Bytes.toBytes("224382618261914241"));
Result result1 = table.get(get1);
System.out.println("Result of get: ");
for (Cell cell : result1.rawCells()) {
System.out.println("Cell: " + cell + ", Value: " +
Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
cell.getValueLength()));
}
table.close();
connection.close();
}
}
/**
Result of columnPrefixFilter get:
Cell: 224382618261914241/info:age/1472196211169/Put/vlen=2/seqid=0, Value: 24
Cell: 224382618261914241/info:height/1472196211234/Put/vlen=3/seqid=0, Value: 158
Cell: 224382618261914241/ship:addr/1472196211487/Put/vlen=7/seqid=0, Value: chengdu
Result of get:
Cell: 224382618261914241/info:age/1472196211169/Put/vlen=2/seqid=0, Value: 24
Cell: 224382618261914241/info:height/1472196211234/Put/vlen=3/seqid=0, Value: 158
Cell: 224382618261914241/info:name/1472196211088/Put/vlen=4/seqid=0, Value: lisi
Cell: 224382618261914241/info:phone/1472196211427/Put/vlen=11/seqid=0, Value: 13213921424
Cell: 224382618261914241/info:weight/1472196211386/Put/vlen=3/seqid=0, Value: 128
Cell: 224382618261914241/ship:addr/1472196211487/Put/vlen=7/seqid=0, Value: chengdu
Cell: 224382618261914241/ship:email/1472196211530/Put/vlen=11/seqid=0, Value: qq@sina.com
Cell: 224382618261914241/ship:salary/1472196211594/Put/vlen=4/seqid=0, Value: 5000
**/
ColumnRangeFilter 列范围过滤器
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.Cell;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.*;
import org.apache.hadoop.hbase.filter.ColumnRangeFilter;
import org.apache.hadoop.hbase.filter.Filter;
import org.apache.hadoop.hbase.util.Bytes;
import java.io.IOException;
/**
* ColumnRangeFilter 列范围过滤器
*/
public class FilterOfColumnRangeFilter {
public static void main(String args[]) throws IOException{
Configuration configuration = HBaseConfiguration.create();
Connection connection = ConnectionFactory.createConnection(configuration);
Table table = connection.getTable(TableName.valueOf(“user”));
//minColumn - minimum value for the column range. If if it’s null, there is no lower bound.
//minColumnInclusive - if true, include minColumn in the range. 如果是true 就要包含minColumn
//maxColumn - maximum value for the column range. If it’s null,
//maxColumnInclusive - if true, include maxColumn in the range. there is no upper bound.
//从email到phone范围内的所有列 第二个参数为true所以包含了email
Filter filter = new ColumnRangeFilter(Bytes.toBytes(“email”), true, Bytes.toBytes(“phone”), false);
Get get = new Get(Bytes.toBytes(“224382618261914241”));
get.setFilter(filter);
Result result = table.get(get);
System.out.println(“Result of ColumnRangeFilter get: “);
for (Cell cell : result.rawCells()) {
System.out.println(“Cell: ” + cell + “, Value: ” +
Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
cell.getValueLength()));
}
Get get1 = new Get(Bytes.toBytes("224382618261914241"));
Result result1 = table.get(get1);
System.out.println("Result of get: ");
for (Cell cell : result1.rawCells()) {
System.out.println("Cell: " + cell + ", Value: " +
Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
cell.getValueLength()));
}
table.close();
connection.close();
}
}
/**
Result of ColumnRangeFilter get:
Cell: 224382618261914241/info:height/1472196211234/Put/vlen=3/seqid=0, Value: 158
Cell: 224382618261914241/info:name/1472196211088/Put/vlen=4/seqid=0, Value: lisi
Cell: 224382618261914241/ship:email/1472196211530/Put/vlen=11/seqid=0, Value: qq@sina.com
Result of get:
Cell: 224382618261914241/info:age/1472196211169/Put/vlen=2/seqid=0, Value: 24
Cell: 224382618261914241/info:height/1472196211234/Put/vlen=3/seqid=0, Value: 158
Cell: 224382618261914241/info:name/1472196211088/Put/vlen=4/seqid=0, Value: lisi
Cell: 224382618261914241/info:phone/1472196211427/Put/vlen=11/seqid=0, Value: 13213921424
Cell: 224382618261914241/info:weight/1472196211386/Put/vlen=3/seqid=0, Value: 128
Cell: 224382618261914241/ship:addr/1472196211487/Put/vlen=7/seqid=0, Value: chengdu
Cell: 224382618261914241/ship:email/1472196211530/Put/vlen=11/seqid=0, Value: qq@sina.com
Cell: 224382618261914241/ship:salary/1472196211594/Put/vlen=4/seqid=0, Value: 5000
2016-08-31 17:53:28,394 INFO [main] client.ConnectionManager$HConnectionImplementati
**/
SingleColumnValueFilter 单列值过滤器
用一列的值决定是否一行数据被过滤
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.Cell;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.*;
import org.apache.hadoop.hbase.filter.CompareFilter;
import org.apache.hadoop.hbase.filter.SingleColumnValueFilter;
import org.apache.hadoop.hbase.filter.SubstringComparator;
import org.apache.hadoop.hbase.util.Bytes;
import java.io.IOException;
/**
* SingleColumnValueFilter 单列过滤器
*/
public class FilterOfSingleColumnValueFilter {
public static void main(String args[]) throws IOException{
Configuration configuration = HBaseConfiguration.create();
Connection connection = ConnectionFactory.createConnection(configuration);
Table table = connection.getTable(TableName.valueOf(“user”));
// 510824118261011172 column=ship:email, timestamp=1472196213422, value=199@sina.com
SingleColumnValueFilter singleColumnValueFilter = new SingleColumnValueFilter(Bytes.toBytes("ship"),Bytes.toBytes("email"), CompareFilter.CompareOp.EQUAL,new SubstringComparator("199@sina.com"));
singleColumnValueFilter.setFilterIfMissing(true);
Scan scan = new Scan();
scan.setFilter(singleColumnValueFilter);
ResultScanner results = table.getScanner(scan);
for (Result result:results){
for (Cell cell :result.rawCells()){
System.out.println("Cell: "+cell+",Value:" + Bytes.toString(cell.getValueArray(),cell.getValueOffset(), cell.getValueLength()));
}
}
results.close();
// 224382618261914241 column=ship:email, timestamp=1472196211530, value=qq@sina.com
Get get = new Get(Bytes.toBytes("224382618261914241"));
get.setFilter(singleColumnValueFilter);
Result result = table.get(get);
System.out.println("Result of get: ");
for (Cell cell : result.rawCells()) {
System.out.println("Cell: " + cell + ", Value: " +
Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
cell.getValueLength()));
}
Get get1 = new Get(Bytes.toBytes("510824118261011172"));
get1.setFilter(singleColumnValueFilter);
Result result1 = table.get(get1);
System.out.println("Result of get1: ");
for (Cell cell : result1.rawCells()) {
System.out.println("Cell: " + cell + ", Value: " +
Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
cell.getValueLength()));
}
table.close();
connection.close();
}
}
/**
Cell: 510824118261011172/info:age/1472196213020/Put/vlen=2/seqid=0,Value:18
Cell: 510824118261011172/info:height/1472196213056/Put/vlen=3/seqid=0,Value:188
Cell: 510824118261011172/info:name/1472196212942/Put/vlen=8/seqid=0,Value:yangyang
Cell: 510824118261011172/info:phone/1472196213237/Put/vlen=11/seqid=0,Value:18013921626
Cell: 510824118261011172/info:weight/1472196213169/Put/vlen=3/seqid=0,Value:138
Cell: 510824118261011172/ship:addr/1472196213328/Put/vlen=8/seqid=0,Value:shanghai
Cell: 510824118261011172/ship:email/1472196213422/Put/vlen=12/seqid=0,Value:199@sina.com
Cell: 510824118261011172/ship:salary/1472196214963/Put/vlen=5/seqid=0,Value:50000
Result of get:
Result of get1:
Cell: 510824118261011172/info:age/1472196213020/Put/vlen=2/seqid=0, Value: 18
Cell: 510824118261011172/info:height/1472196213056/Put/vlen=3/seqid=0, Value: 188
Cell: 510824118261011172/info:name/1472196212942/Put/vlen=8/seqid=0, Value: yangyang
Cell: 510824118261011172/info:phone/1472196213237/Put/vlen=11/seqid=0, Value: 18013921626
Cell: 510824118261011172/info:weight/1472196213169/Put/vlen=3/seqid=0, Value: 138
Cell: 510824118261011172/ship:addr/1472196213328/Put/vlen=8/seqid=0, Value: shanghai
Cell: 510824118261011172/ship:email/1472196213422/Put/vlen=12/seqid=0, Value: 199@sina.com
Cell: 510824118261011172/ship:salary/1472196214963/Put/vlen=5/seqid=0, Value: 50000
**/
SingleColumnValueExcludeFilter 单列排除过滤器
单列排除过滤器继承自SingleColumnValueFilter,过滤的方式还是按照SingleColumnValueFilter去过滤,
但是最后的结果集去除了作为过滤条件的列
// 510824118261011172 column=ship:email, timestamp=1472196213422, value=199@sina.com
SingleColumnValueExcludeFilter singleColumnValueExcludeFilter = new SingleColumnValueExcludeFilter(Bytes.toBytes("ship"),Bytes.toBytes("email"), CompareFilter.CompareOp.EQUAL,new SubstringComparator("199@sina.com"));
singleColumnValueExcludeFilter.setFilterIfMissing(true);
Scan scan = new Scan();
scan.setFilter(singleColumnValueExcludeFilter);
ResultScanner results = table.getScanner(scan);
for (Result result:results){
for (Cell cell :result.rawCells()){
System.out.println("Cell: "+cell+",Value:" + Bytes.toString(cell.getValueArray(),cell.getValueOffset(), cell.getValueLength()));
}
}
results.close();
// 224382618261914241 column=ship:email, timestamp=1472196211530, value=qq@sina.com
Get get = new Get(Bytes.toBytes("224382618261914241"));
get.setFilter(singleColumnValueExcludeFilter);
Result result = table.get(get);
System.out.println("Result of get: ");
for (Cell cell : result.rawCells()) {
System.out.println("Cell: " + cell + ", Value: " +
Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
cell.getValueLength()));
}
Get get1 = new Get(Bytes.toBytes("510824118261011172"));
get1.setFilter(singleColumnValueExcludeFilter);
Result result1 = table.get(get1);
System.out.println("Result of get1: ");
for (Cell cell : result1.rawCells()) {
System.out.println("Cell: " + cell + ", Value: " +
Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
cell.getValueLength()));
}
/**
* Cell: 510824118261011172/info:age/1472196213020/Put/vlen=2/seqid=0,Value:18
Cell: 510824118261011172/info:height/1472196213056/Put/vlen=3/seqid=0,Value:188
Cell: 510824118261011172/info:name/1472196212942/Put/vlen=8/seqid=0,Value:yangyang
Cell: 510824118261011172/info:phone/1472196213237/Put/vlen=11/seqid=0,Value:18013921626
Cell: 510824118261011172/info:weight/1472196213169/Put/vlen=3/seqid=0,Value:138
Cell: 510824118261011172/ship:addr/1472196213328/Put/vlen=8/seqid=0,Value:shanghai
Cell: 510824118261011172/ship:salary/1472196214963/Put/vlen=5/seqid=0,Value:50000
Result of get:
Result of get1:
Cell: 510824118261011172/info:age/1472196213020/Put/vlen=2/seqid=0, Value: 18
Cell: 510824118261011172/info:height/1472196213056/Put/vlen=3/seqid=0, Value: 188
Cell: 510824118261011172/info:name/1472196212942/Put/vlen=8/seqid=0, Value: yangyang
Cell: 510824118261011172/info:phone/1472196213237/Put/vlen=11/seqid=0, Value: 18013921626
Cell: 510824118261011172/info:weight/1472196213169/Put/vlen=3/seqid=0, Value: 138
Cell: 510824118261011172/ship:addr/1472196213328/Put/vlen=8/seqid=0, Value: shanghai
Cell: 510824118261011172/ship:salary/1472196214963/Put/vlen=5/seqid=0, Value: 50000
*/
TimestampsFilter 时间过滤器
使用时间戳的值来过滤值
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.*;
import org.apache.hadoop.hbase.filter.Filter;
import org.apache.hadoop.hbase.filter.TimestampsFilter;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
/**
* TimestampsFilter 时间过滤器
*/
public class FilterOfTimestampsFilter {
public static void main(String args[]) throws IOException{
Configuration configuration = HBaseConfiguration.create();
Connection connection = ConnectionFactory.createConnection(configuration);
Table table = connection.getTable(TableName.valueOf("user"));
List<Long> ts = new ArrayList<Long>();
ts.add(new Long("1472196195270"));
ts.add(new Long("1472196212480"));
ts.add(new Long(15));
Filter filter = new TimestampsFilter(ts);
Scan scan1 = new Scan();
scan1.setFilter(filter);
ResultScanner scanner1 = table.getScanner(scan1);
for(Result result:scanner1){
System.out.println(result);
}
scanner1.close();
Scan scan2 = new Scan();
scan2.setFilter(filter);
//加了时间范围 故意多加了1s 1472196212480+1=1472196212481
scan2.setTimeRange(1472196195271L, 1472196212481L);
ResultScanner scanner2 = table.getScanner(scan2);
System.out.println("Add time range:");
for (Result result : scanner2) {
System.out.println(result);
}
scanner2.close();
}
}
/**
keyvalues={524382618264914241/ship:addr/1472196195270/Put/vlen=7/seqid=0}
keyvalues={813782218261011172/info:name/1472196212480/Put/vlen=8/seqid=0}
Add time range:
keyvalues={813782218261011172/info:name/1472196212480/Put/vlen=8/seqid=0}
**/
RandomRowFilter 随机行过滤器
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.Cell;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.*;
import org.apache.hadoop.hbase.filter.Filter;
import org.apache.hadoop.hbase.filter.RandomRowFilter;
import org.apache.hadoop.hbase.util.Bytes;
import java.io.IOException;
/**
* RandomRowFilter 随机行过滤器
*/
public class FilterOfRandomRowFilter {
public static void main(String args[]) throws IOException{
Configuration configuration = HBaseConfiguration.create();
Connection connection = ConnectionFactory.createConnection(configuration);
Table table = connection.getTable(TableName.valueOf(“user”));
//该构造参数0-1之间,如果为负数全部过滤,大于1全部通过 0.2f表的该行数据20%的概率通过
Filter filter = new RandomRowFilter(0.2f);
Scan scan = new Scan();
scan.setFilter(filter);
ResultScanner results = table.getScanner(scan);
for (Result result:results) {
for (Cell cell : result.rawCells()) {
System.out.println(“Cell: ” + cell + “, Value: ” +
Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
cell.getValueLength()));
}
}
table.close();
connection.close();
}
}
/**
Cell: 510824118261011172/info:age/1472196213020/Put/vlen=2/seqid=0, Value: 18
Cell: 510824118261011172/info:height/1472196213056/Put/vlen=3/seqid=0, Value: 188
Cell: 510824118261011172/info:name/1472196212942/Put/vlen=8/seqid=0, Value: yangyang
Cell: 510824118261011172/info:phone/1472196213237/Put/vlen=11/seqid=0, Value: 18013921626
Cell: 510824118261011172/info:weight/1472196213169/Put/vlen=3/seqid=0, Value: 138
Cell: 510824118261011172/ship:addr/1472196213328/Put/vlen=8/seqid=0, Value: shanghai
Cell: 510824118261011172/ship:email/1472196213422/Put/vlen=12/seqid=0, Value: 199@sina.com
Cell: 510824118261011172/ship:salary/1472196214963/Put/vlen=5/seqid=0, Value: 50000
**/
Decorating Filters 装饰过滤器或附加过滤器
SkipFilter 跳过过滤器 根据构造器中的过滤器为基准跳过行
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.Cell;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.*;
import org.apache.hadoop.hbase.filter.*;
import org.apache.hadoop.hbase.util.Bytes;
import java.io.IOException;
/**
* SkipFilter 跳过(忽略)过滤器
* similarface
* similarface@outlook.com
*/
public class FilterOfSkipFilter {
public static void main(String args[]) throws IOException{
Configuration configuration = HBaseConfiguration.create();
Connection connection = ConnectionFactory.createConnection(configuration);
Table table = connection.getTable(TableName.valueOf(“user”));
//510824118261011172 column=info:height, timestamp=1472196213056, value=188
//673782618261019142 column=info:weight, timestamp=1472196211841, value=188
//如果列值中含有188这个数值,那么这列将会跳过
Filter filter = new ValueFilter(CompareFilter.CompareOp.NOT_EQUAL,new BinaryComparator(Bytes.toBytes(“188”)));
Scan scan = new Scan();
scan.setFilter(filter);
ResultScanner results = table.getScanner(scan);
for (Result result:results) {
for (Cell cell : result.rawCells()) {
System.out.println("Cell: " + cell + ", Value: " +
Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
cell.getValueLength()));
}
}
Scan scan1 = new Scan();
//skipfilter的构造参数时filter
// filter==>如果列值中含有188这个数值,那么这列将会跳过 filter2==>就是如果列值中含有188这个数值那么整个行都会被跳过 表示不会出现[510824118261011172,673782618261019142]
Filter filter2 = new SkipFilter(filter);
scan1.setFilter(filter2);
ResultScanner scanner2= table.getScanner(scan1);
for(Result result:scanner2){
for (Cell cell : result.rawCells()) {
System.out.println("SKIP Cell: " + cell + ", Value: " +
Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
cell.getValueLength()));
}
}
results.close();
scanner2.close();
table.close();
connection.close();
}
}
/**
Cell: 224382618261914241/info:age/1472196211169/Put/vlen=2/seqid=0, Value: 24
Cell: 224382618261914241/info:height/1472196211234/Put/vlen=3/seqid=0, Value: 158
Cell: 224382618261914241/info:name/1472196211088/Put/vlen=4/seqid=0, Value: lisi
Cell: 224382618261914241/info:phone/1472196211427/Put/vlen=11/seqid=0, Value: 13213921424
Cell: 224382618261914241/info:weight/1472196211386/Put/vlen=3/seqid=0, Value: 128
Cell: 224382618261914241/ship:addr/1472196211487/Put/vlen=7/seqid=0, Value: chengdu
Cell: 224382618261914241/ship:email/1472196211530/Put/vlen=11/seqid=0, Value: qq@sina.com
Cell: 224382618261914241/ship:salary/1472196211594/Put/vlen=4/seqid=0, Value: 5000
Cell: 510824118261011172/info:age/1472196213020/Put/vlen=2/seqid=0, Value: 18
Cell: 510824118261011172/info:name/1472196212942/Put/vlen=8/seqid=0, Value: yangyang
Cell: 510824118261011172/info:phone/1472196213237/Put/vlen=11/seqid=0, Value: 18013921626
Cell: 510824118261011172/info:weight/1472196213169/Put/vlen=3/seqid=0, Value: 138
Cell: 510824118261011172/ship:addr/1472196213328/Put/vlen=8/seqid=0, Value: shanghai
Cell: 510824118261011172/ship:email/1472196213422/Put/vlen=12/seqid=0, Value: 199@sina.com
Cell: 510824118261011172/ship:salary/1472196214963/Put/vlen=5/seqid=0, Value: 50000
Cell: 524382618264914241/info:age/1472196193913/Put/vlen=2/seqid=0, Value: 30
Cell: 524382618264914241/info:height/1472196194783/Put/vlen=3/seqid=0, Value: 168
Cell: 524382618264914241/info:name/1472196193255/Put/vlen=8/seqid=0, Value: zhangsan
Cell: 524382618264914241/info:phone/1472196195125/Put/vlen=11/seqid=0, Value: 13212321424
Cell: 524382618264914241/info:weight/1472196194970/Put/vlen=3/seqid=0, Value: 168
Cell: 524382618264914241/ship:addr/1472196195270/Put/vlen=7/seqid=0, Value: beijing
Cell: 524382618264914241/ship:email/1472196195371/Put/vlen=13/seqid=0, Value: sina@sina.com
Cell: 524382618264914241/ship:salary/1472196195485/Put/vlen=4/seqid=0, Value: 3000
Cell: 673782618261019142/info:age/1472196211733/Put/vlen=2/seqid=0, Value: 19
Cell: 673782618261019142/info:height/1472196211761/Put/vlen=3/seqid=0, Value: 178
Cell: 673782618261019142/info:name/1472196211678/Put/vlen=7/seqid=0, Value: zhaoliu
Cell: 673782618261019142/info:phone/1472196211956/Put/vlen=11/seqid=0, Value: 17713921424
Cell: 673782618261019142/ship:addr/1472196212059/Put/vlen=8/seqid=0, Value: shenzhen
Cell: 673782618261019142/ship:email/1472196212176/Put/vlen=12/seqid=0, Value: 126@sina.com
Cell: 673782618261019142/ship:salary/1472196212284/Put/vlen=4/seqid=0, Value: 8000
Cell: 813782218261011172/info:age/1472196212550/Put/vlen=2/seqid=0, Value: 19
Cell: 813782218261011172/info:height/1472196212605/Put/vlen=3/seqid=0, Value: 158
Cell: 813782218261011172/info:name/1472196212480/Put/vlen=8/seqid=0, Value: wangmazi
Cell: 813782218261011172/info:phone/1472196212713/Put/vlen=11/seqid=0, Value: 12713921424
Cell: 813782218261011172/info:weight/1472196212651/Put/vlen=3/seqid=0, Value: 118
Cell: 813782218261011172/ship:addr/1472196212762/Put/vlen=4/seqid=0, Value: xian
Cell: 813782218261011172/ship:email/1472196212802/Put/vlen=12/seqid=0, Value: 139@sina.com
Cell: 813782218261011172/ship:salary/1472196212840/Put/vlen=5/seqid=0, Value: 10000
SKIP Cell: 224382618261914241/info:age/1472196211169/Put/vlen=2/seqid=0, Value: 24
SKIP Cell: 224382618261914241/info:height/1472196211234/Put/vlen=3/seqid=0, Value: 158
SKIP Cell: 224382618261914241/info:name/1472196211088/Put/vlen=4/seqid=0, Value: lisi
SKIP Cell: 224382618261914241/info:phone/1472196211427/Put/vlen=11/seqid=0, Value: 13213921424
SKIP Cell: 224382618261914241/info:weight/1472196211386/Put/vlen=3/seqid=0, Value: 128
SKIP Cell: 224382618261914241/ship:addr/1472196211487/Put/vlen=7/seqid=0, Value: chengdu
SKIP Cell: 224382618261914241/ship:email/1472196211530/Put/vlen=11/seqid=0, Value: qq@sina.com
SKIP Cell: 224382618261914241/ship:salary/1472196211594/Put/vlen=4/seqid=0, Value: 5000
SKIP Cell: 524382618264914241/info:age/1472196193913/Put/vlen=2/seqid=0, Value: 30
SKIP Cell: 524382618264914241/info:height/1472196194783/Put/vlen=3/seqid=0, Value: 168
SKIP Cell: 524382618264914241/info:name/1472196193255/Put/vlen=8/seqid=0, Value: zhangsan
SKIP Cell: 524382618264914241/info:phone/1472196195125/Put/vlen=11/seqid=0, Value: 13212321424
SKIP Cell: 524382618264914241/info:weight/1472196194970/Put/vlen=3/seqid=0, Value: 168
SKIP Cell: 524382618264914241/ship:addr/1472196195270/Put/vlen=7/seqid=0, Value: beijing
SKIP Cell: 524382618264914241/ship:email/1472196195371/Put/vlen=13/seqid=0, Value: sina@sina.com
SKIP Cell: 524382618264914241/ship:salary/1472196195485/Put/vlen=4/seqid=0, Value: 3000
SKIP Cell: 813782218261011172/info:age/1472196212550/Put/vlen=2/seqid=0, Value: 19
SKIP Cell: 813782218261011172/info:height/1472196212605/Put/vlen=3/seqid=0, Value: 158
SKIP Cell: 813782218261011172/info:name/1472196212480/Put/vlen=8/seqid=0, Value: wangmazi
SKIP Cell: 813782218261011172/info:phone/1472196212713/Put/vlen=11/seqid=0, Value: 12713921424
SKIP Cell: 813782218261011172/info:weight/1472196212651/Put/vlen=3/seqid=0, Value: 118
SKIP Cell: 813782218261011172/ship:addr/1472196212762/Put/vlen=4/seqid=0, Value: xian
SKIP Cell: 813782218261011172/ship:email/1472196212802/Put/vlen=12/seqid=0, Value: 139@sina.com
SKIP Cell: 813782218261011172/ship:salary/1472196212840/Put/vlen=5/seqid=0, Value: 10000
**/
WhileMatchFilter 当匹配到就中断扫描
61
62
63
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.Cell;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.*;
import org.apache.hadoop.hbase.filter.*;
import org.apache.hadoop.hbase.util.Bytes;
import java.io.IOException;
/**
* WhileMatchFilter 全匹配过滤器 [当匹配到就中断扫描]
* similarface
* similarface@outlook.com
*/
public class FilterOfWhileMatchFilter {
public static void main(String args[]) throws IOException{
Configuration configuration = HBaseConfiguration.create();
Connection connection = ConnectionFactory.createConnection(configuration);
Table table = connection.getTable(TableName.valueOf(“user”));
//匹配 510824118261011172 的行
Filter filter1=new RowFilter(CompareFilter.CompareOp.NOT_EQUAL,new BinaryComparator(Bytes.toBytes(“510824118261011172”)));
Scan scan = new Scan();
scan.setFilter(filter1);
ResultScanner results = table.getScanner(scan);
for (Result result:results) {
for (Cell cell : result.rawCells()) {
System.out.println(“Cell: ” + cell + “, Value: ” +
Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
cell.getValueLength()));
}
}
// 匹配 510824118261011172 的行马上中断本次扫描操作
Filter filter2 = new WhileMatchFilter(filter1);
Scan scan1 = new Scan();
scan1.setFilter(filter2);
ResultScanner scanner2= table.getScanner(scan1);
for(Result result:scanner2){
for (Cell cell : result.rawCells()) {
System.out.println(“WhileMatchFilter Cell: ” + cell + “, Value: ” +
Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
cell.getValueLength()));
}
}
results.close();
scanner2.close();
table.close();
connection.close();
}
}
/**
Cell: 224382618261914241/info:age/1472196211169/Put/vlen=2/seqid=0, Value: 24
Cell: 224382618261914241/info:height/1472196211234/Put/vlen=3/seqid=0, Value: 158
…
Cell: 524382618264914241/ship:addr/1472196195270/Put/vlen=7/seqid=0, Value: beijing
Cell: 524382618264914241/ship:email/1472196195371/Put/vlen=13/seqid=0, Value: sina@sina.com
…
Cell: 813782218261011172/ship:salary/1472196212840/Put/vlen=5/seqid=0, Value: 10000
WhileMatchFilter Cell: 224382618261914241/info:age/1472196211169/Put/vlen=2/seqid=0, Value: 24
…
WhileMatchFilter Cell: 224382618261914241/ship:salary/1472196211594/Put/vlen=4/seqid=0, Value: 5000
**/
FilterList 过滤列表[多个过滤器一起起作用]
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.Cell;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.*;
import org.apache.hadoop.hbase.filter.*;
import org.apache.hadoop.hbase.util.Bytes;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
/**
* FilterList 过滤器列表[多个过滤器组合在一起]
* similarface
* similarface@outlook.com
*/
public class FilterOfFilterList {
public static void main(String args[]) throws IOException{
Configuration configuration = HBaseConfiguration.create();
Connection connection = ConnectionFactory.createConnection(configuration);
Table table = connection.getTable(TableName.valueOf(“user”));
List filters = new ArrayList();
Filter filter1=new RowFilter(CompareFilter.CompareOp.GREATER_OR_EQUAL,
new BinaryComparator(Bytes.toBytes(“524382618264914241”)));
filters.add(filter1);
Filter filter2=new RowFilter(CompareFilter.CompareOp.LESS_OR_EQUAL,
new BinaryComparator(Bytes.toBytes("813782218261011172")));
filters.add(filter2);
//列名过滤器
Filter filter3=new QualifierFilter(CompareFilter.CompareOp.EQUAL,
new RegexStringComparator("age"));
filters.add(filter3);
//行键 Between "524382618264914241" and "813782218261011172" and column="age"
FilterList filterList1 = new FilterList(filters);
Scan scan = new Scan();
scan.setFilter(filterList1);
ResultScanner results = table.getScanner(scan);
for (Result result:results) {
for (Cell cell : result.rawCells()) {
System.out.println("Cell: " + cell + ", Value: " +
Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
cell.getValueLength()));
}
}
results.close();
//行键 Between "524382618264914241" or "813782218261011172" or column="age"
FilterList filterList2 = new FilterList(FilterList.Operator.MUST_PASS_ONE,filters);
scan.setFilter(filterList2);
ResultScanner scanner2= table.getScanner(scan);
for(Result result:scanner2){
for (Cell cell : result.rawCells()) {
System.out.println("MUST_PASS_ONE Cell: " + cell + ", Value: " +
Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
cell.getValueLength()));
}
}
scanner2.close();
table.close();
connection.close();
}
}
/**
Cell: 524382618264914241/info:age/1472196193913/Put/vlen=2/seqid=0, Value: 30
Cell: 673782618261019142/info:age/1472196211733/Put/vlen=2/seqid=0, Value: 19
Cell: 813782218261011172/info:age/1472196212550/Put/vlen=2/seqid=0, Value: 19
MUST_PASS_ONE Cell: 224382618261914241/info:age/1472196211169/Put/vlen=2/seqid=0, Value: 24
MUST_PASS_ONE Cell: 224382618261914241/info:height/1472196211234/Put/vlen=3/seqid=0, Value: 158
MUST_PASS_ONE Cell: 224382618261914241/info:name/1472196211088/Put/vlen=4/seqid=0, Value: lisi
MUST_PASS_ONE Cell: 224382618261914241/info:phone/1472196211427/Put/vlen=11/seqid=0, Value: 13213921424
MUST_PASS_ONE Cell: 224382618261914241/info:weight/1472196211386/Put/vlen=3/seqid=0, Value: 128
MUST_PASS_ONE Cell: 224382618261914241/ship:addr/1472196211487/Put/vlen=7/seqid=0, Value: chengdu
MUST_PASS_ONE Cell: 224382618261914241/ship:email/1472196211530/Put/vlen=11/seqid=0, Value: qq@sina.com
MUST_PASS_ONE Cell: 224382618261914241/ship:salary/1472196211594/Put/vlen=4/seqid=0, Value: 5000
MUST_PASS_ONE Cell: 510824118261011172/info:age/1472196213020/Put/vlen=2/seqid=0, Value: 18
MUST_PASS_ONE Cell: 510824118261011172/info:height/1472196213056/Put/vlen=3/seqid=0, Value: 188
MUST_PASS_ONE Cell: 510824118261011172/info:name/1472196212942/Put/vlen=8/seqid=0, Value: yangyang
MUST_PASS_ONE Cell: 510824118261011172/info:phone/1472196213237/Put/vlen=11/seqid=0, Value: 18013921626
MUST_PASS_ONE Cell: 510824118261011172/info:weight/1472196213169/Put/vlen=3/seqid=0, Value: 138
MUST_PASS_ONE Cell: 510824118261011172/ship:addr/1472196213328/Put/vlen=8/seqid=0, Value: shanghai
MUST_PASS_ONE Cell: 510824118261011172/ship:email/1472196213422/Put/vlen=12/seqid=0, Value: 199@sina.com
MUST_PASS_ONE Cell: 510824118261011172/ship:salary/1472196214963/Put/vlen=5/seqid=0, Value: 50000
MUST_PASS_ONE Cell: 524382618264914241/info:age/1472196193913/Put/vlen=2/seqid=0, Value: 30
MUST_PASS_ONE Cell: 524382618264914241/info:height/1472196194783/Put/vlen=3/seqid=0, Value: 168
MUST_PASS_ONE Cell: 524382618264914241/info:name/1472196193255/Put/vlen=8/seqid=0, Value: zhangsan
MUST_PASS_ONE Cell: 524382618264914241/info:phone/1472196195125/Put/vlen=11/seqid=0, Value: 13212321424
MUST_PASS_ONE Cell: 524382618264914241/info:weight/1472196194970/Put/vlen=3/seqid=0, Value: 168
MUST_PASS_ONE Cell: 524382618264914241/ship:addr/1472196195270/Put/vlen=7/seqid=0, Value: beijing
MUST_PASS_ONE Cell: 524382618264914241/ship:email/1472196195371/Put/vlen=13/seqid=0, Value: sina@sina.com
MUST_PASS_ONE Cell: 524382618264914241/ship:salary/1472196195485/Put/vlen=4/seqid=0, Value: 3000
MUST_PASS_ONE Cell: 673782618261019142/info:age/1472196211733/Put/vlen=2/seqid=0, Value: 19
MUST_PASS_ONE Cell: 673782618261019142/info:height/1472196211761/Put/vlen=3/seqid=0, Value: 178
MUST_PASS_ONE Cell: 673782618261019142/info:name/1472196211678/Put/vlen=7/seqid=0, Value: zhaoliu
MUST_PASS_ONE Cell: 673782618261019142/info:phone/1472196211956/Put/vlen=11/seqid=0, Value: 17713921424
MUST_PASS_ONE Cell: 673782618261019142/info:weight/1472196211841/Put/vlen=3/seqid=0, Value: 188
MUST_PASS_ONE Cell: 673782618261019142/ship:addr/1472196212059/Put/vlen=8/seqid=0, Value: shenzhen
MUST_PASS_ONE Cell: 673782618261019142/ship:email/1472196212176/Put/vlen=12/seqid=0, Value: 126@sina.com
MUST_PASS_ONE Cell: 673782618261019142/ship:salary/1472196212284/Put/vlen=4/seqid=0, Value: 8000
MUST_PASS_ONE Cell: 813782218261011172/info:age/1472196212550/Put/vlen=2/seqid=0, Value: 19
MUST_PASS_ONE Cell: 813782218261011172/info:height/1472196212605/Put/vlen=3/seqid=0, Value: 158
MUST_PASS_ONE Cell: 813782218261011172/info:name/1472196212480/Put/vlen=8/seqid=0, Value: wangmazi
MUST_PASS_ONE Cell: 813782218261011172/info:phone/1472196212713/Put/vlen=11/seqid=0, Value: 12713921424
MUST_PASS_ONE Cell: 813782218261011172/info:weight/1472196212651/Put/vlen=3/seqid=0, Value: 118
MUST_PASS_ONE Cell: 813782218261011172/ship:addr/1472196212762/Put/vlen=4/seqid=0, Value: xian
MUST_PASS_ONE Cell: 813782218261011172/ship:email/1472196212802/Put/vlen=12/seqid=0, Value: 139@sina.com
MUST_PASS_ONE Cell: 813782218261011172/ship:salary/1472196212840/Put/vlen=5/seqid=0, Value: 10000
**/
Custom Filters
Custom Filter Loading