hadoop复合键排序使用方法

在Hadoop中处理复杂业务时,需要用到复合键,复合不同于单纯的继承Writable接口,而是继承了WritableComparable接口,而实际上,WritableComparable接口继承了Writable和Comparable接口,如果只需要使用某一个类作为传值对象而不是作为key,继承Writable接口即可。
public interface WritableComparable extends Writable, Comparable {
}

public interface Writable {

void write(DataOutput out) throws IOException;

void readFields(DataInput in) throws IOException;
}

public interface Comparable {

public int compareTo(T o);  

}
以下是实现复合key的实例,亲测,可用
[java] view plain copy
print?
public class SortKey implements WritableComparable{

private Text name;  
private IntWritable right;  


public SortKey() {  
    set(new Text(), new IntWritable());  
}  


public SortKey(Text name, IntWritable right) {  
    set(name, right);  
}  


private void set(Text name,IntWritable right){  
    this.name = name;  
    this.right = right;  
}  



/** 
 * @return the name 
 */  
public Text getName() {  
    return name;  
}  


/** 
 * @param name the name to set 
 */  
public void setName(Text name) {  
    this.name = name;  
}  


/** 
 * @return the right 
 */  
public IntWritable getRight() {  
    return right;  
}  


/** 
 * @param right the right to set 
 */  
public void setRight(IntWritable right) {  
    this.right = right;  
}  


@Override  
public void write(DataOutput out) throws IOException {  
    name.write(out);  
    right.write(out);  
}  

@Override  
public void readFields(DataInput in) throws IOException {  
    name.readFields(in);  
    right.readFields(in);  
}  

@Override  
public int compareTo(SortKey o) {  
    int cmp = name.compareTo(o.name);  
    if(cmp != 0){  
        return cmp;  
    }else{  
        return right.compareTo(o.right);  
    }  
}  

//到目前为止,你只能将其作为key来使用,但是如果你需要按照key的某一个值来排序,以下是重点
static{
WritableComparator.define(SortKey.class, new Comparator());
}

public static class Comparator extends WritableComparator{  

    private static final Text.Comparator TEXT_COMPARATOR = new Text.Comparator();  

    protected Comparator() {  
        super(SortKey.class);  
    }  

    /* (non-Javadoc) 
     * @see org.apache.hadoop.io.WritableComparator#compare(byte[], int, int, byte[], int, int) 
     */  
    @Override  
    public int compare(byte[] b1, int s1, int l1, byte[] b2, int s2, int l2) {  
        try{  
            int firstL1 = WritableUtils.decodeVIntSize(b1[s1]) + readVInt(b1, s1);  
            int firstL2 = WritableUtils.decodeVIntSize(b2[s2]) + readVInt(b2, s2);  
            return TEXT_COMPARATOR.compare(b1, s1, firstL1, b2, s2, firstL2);  
        }catch(Exception e){  
            throw new IllegalArgumentException(e);  
        }  
    }     
}  

}

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值