关闭

secondary sort

512人阅读 评论(0) 收藏 举报
class FirstPartitioner extends Partitioner<Text, MapWritable> {
    @Override
    public int getPartition(Text key, MapWritable value, int numPartitions) {
      String []allKey = key.toString().split("\t");
      if(allKey.length<2){
        throw new RuntimeException("key's indicator does not exist.");
      }
      String url = allKey[0];
      return Math.abs(url.hashCode()) % numPartitions;
    }


  }


   class KeyComparator extends WritableComparator {
    protected KeyComparator() {
      super(Text.class, true);
    }
    @Override
    public int compare(WritableComparable w1, WritableComparable w2) {
      Text ip1 = (Text) w1;
      Text ip2 = (Text) w2;
      int cmp = ip1.toString().split("\t")[0].compareTo(ip2.toString().split("\t")[0]);
      if (cmp != 0) {
        return cmp;
      }
      return -ip1.toString().split("\t")[1].compareTo(ip2.toString().split("\t")[1]); //reverse
    }
  }


  class GroupComparator extends WritableComparator {
    protected GroupComparator() {
      super(Text.class, true);
    }
    @Override
    public int compare(WritableComparable w1, WritableComparable w2) {
      Text ip1 = (Text) w1;
      Text ip2 = (Text) w2;
      return ip1.toString().split("\t")[0].compareTo(ip2.toString().split("\t")[0]);
    }
  }

0
0

查看评论
* 以上用户言论只代表其个人观点,不代表CSDN网站的观点或立场
    个人资料
    • 访问:737516次
    • 积分:10743
    • 等级:
    • 排名:第1498名
    • 原创:417篇
    • 转载:86篇
    • 译文:2篇
    • 评论:106条
    最新评论