最近在学习MapReduce时,找了一些小案例练习,案例中需要map输出和分组时都实现自定义比较规则,于是继承了WritableComparator类,并重写了compare方法,但发现自定义的比较规则始终没有生效,最终发现是重写的方法错了。
注意WritableComparator有多种compare方法,如下:
@Override
public int compare(byte[] b1, int s1, int l1, byte[] b2, int s2, int l2) {
try {
buffer.reset(b1, s1, l1); // parse key1
key1.readFields(buffer);
buffer.reset(b2, s2, l2); // parse key2
key2.readFields(buffer);
} catch (IOException e) {
throw new RuntimeException(e);
}
return compare(key1, key2); // compare them
}
/** Compare two WritableComparables.
*
* <p> The default implementation uses the natural ordering, calling {@link
* Comparable#compareTo(Object)}. */
@SuppressWarnings("unchecked")
public int compare(WritableComparable a, WritableComparable b) {
return a.compareTo(b);
}
@Override
public int compare(Object a, Object b) {
return compare((WritableComparable)a, (WritableComparable)b);
}
一开始没注意重写的方法入参是Object的,导致排序始终不生效
public int compare(Object a, Object b)
compare方法最终也是调用compare(WritableComparable a, WritableComparable b) 实现排序,所以应该要重写方法入参是WritableComparable 的方法
public int compare(WritableComparable a, WritableComparable b)