自定义排序

目录

实现WritableComparable接口

重写compareTo方法

SaleSortMapper

SaleSortReducer

SaleSortDrive

---

 在Hadoop MapReduce框架中，当Mapper的输出键（Key）或者Reducer的输入及输出键上实现WritableComparable接口时，框架会自动利用该接口的compareTo方法对键进行排序。

 事例入口文件参考

mapreduce实现bean的序列化与反序列化

实现WritableComparable接口

public class UserSaleSort implements WritableComparable<UserSaleSort> {

 

//销售id
private int saleId;
//用户名称
private String username;
//用户性别
private String sex;
//商品名称
private String goodsName;
//商品单价
private  int price;
//购买数量
private  int saleCount;
//购买总价
private  int totalPrice;

重写compareTo方法

@Override
public int compareTo(UserSaleSort userSaleSort) {

    //获取被比较的对象的订单总额

    int totalPriceCompare = userSaleSort.getTotalPrice();
    int returnNum = 0;
    if(this.totalPrice<totalPriceCompare){
        returnNum = -1;
    } else if (this.totalPrice>totalPriceCompare) {
        returnNum = 1;
    }else {
        returnNum = 0;
    }
    return  returnNum;
}

序列化，反序列化 

 

//重写序列化方法
    @Override
    public void write(DataOutput out) throws IOException {
    out.writeInt(saleId);
    out.writeUTF(username);
    out.writeUTF(sex);
    out.writeUTF(goodsName);
    out.writeInt(price);
    out.writeInt(saleCount);
    out.writeInt(totalPrice);
    }
    //重写反序列化
    @Override
    public void readFields(DataInput in) throws IOException {
    this.saleId = in.readInt();
    this.username = in.readUTF();
    this.sex = in.readUTF();
    this.goodsName = in.readUTF();
    this.price = in.readInt();
    this.saleCount = in.readInt();
    this.totalPrice = in.readInt();
    }

 以及构造函数，get，set方法等

 

//重写toString方法
    //最终输出到文件的value
    @Override
    public String toString() {
        return " "+totalPrice;
    }

    public int getSaleId() {
        return saleId;
    }

    public void setSaleId(int saleId) {
        this.saleId = saleId;
    }

    public String getUsername() {
        return username;
    }

    public void setUsername(String username) {
        this.username = username;
    }

    public String getSex() {
        return sex;
    }

    public void setSex(String sex) {
        this.sex = sex;
    }

    public String getGoodsName() {
        return goodsName;
    }

    public void setGoodsName(String goodsName) {
        this.goodsName = goodsName;
    }

    public int getPrice() {
        return price;
    }

    public void setPrice(int price) {
        this.price = price;
    }

    public int getSaleCount() {
        return saleCount;
    }

    public void setSaleCount(int saleCount) {
        this.saleCount = saleCount;
    }

    public int getTotalPrice() {
        return totalPrice;
    }

    public void setTotalPrice(int totalPrice) {
        this.totalPrice = totalPrice;
    }
    public void setTotalPrice(){
        this.totalPrice = this.price*this.saleCount;
    }

 

SaleSortMapper

public class SaleSortMapper extends Mapper<LongWritable, Text, UserSaleSort,Text>{
    //根据默认排序是根据map阶段输出的key进行排序，所以，key换成我们有自定义排序的UserSaleSort对象
   //创建输出key对象
    UserSaleSort keyOut = new UserSaleSort();

    //创建输出value
    private Text valueOut = new Text();

    @Override
    protected void map(LongWritable key, Text value, Mapper<LongWritable, Text, UserSaleSort, Text>.Context context) throws IOException, InterruptedException {
        //获取一行数据
        String line = value.toString();
        //根据分隔符拆解数据
        String[] saleDetails = line.split(",");
        //封装对象
        keyOut.setSaleId(Integer.parseInt(saleDetails[0]));
        keyOut.setUsername(saleDetails[1]);
        keyOut.setSex(saleDetails[2]);
        keyOut.setGoodsName(saleDetails[3]);
        keyOut.setPrice(Integer.parseInt(saleDetails[4]));
        keyOut.setSaleCount(Integer.parseInt(saleDetails[5]));


        //赋值
        valueOut.set(keyOut.getUsername());
        //计算总价
        keyOut.setTotalPrice();
        System.out.println(keyOut+"+"+valueOut);
        //map阶段的输出
        context.write(keyOut,valueOut);

    }
}

 Mapper<LongWritable, Text, UserSaleSort,Text>

这里的Map阶段输出的键是UserSaleSort对象，因为当Mapper的输出键（Key)上实现WritableComparable接口时，框架会自动利用该接口的compareTo方法对键进行排序。

SaleSortReducer

public class SaleSortReducer extends Reducer<UserSaleSort,Text, Text, UserSaleSort> {
    private Text keyOut = new Text();
    @Override
    protected void reduce(UserSaleSort key, Iterable<Text> values, Reducer<UserSaleSort, Text, Text, UserSaleSort>.Context context) throws IOException, InterruptedException {

        //此时迭代器里的是Text，用户名，只有用户名就没有迭代的意义了

        keyOut.set(key.getUsername());

        //直接输出

        context.write(keyOut,key);
    }
}

 Reducer<UserSaleSort,Text, Text, UserSaleSort>

和map阶段输出键值一样

此时迭代器里的是Text（Iterable<Text>），用户名，只有用户名就没有迭代的意义了

SaleSortDrive

public class SaleSortDriver {
    public static void main(String[] args) throws IOException, InterruptedException, ClassNotFoundException {
        //1.获取配置对象和job对象
        Configuration conf = new Configuration();
        Job job = Job.getInstance(conf);
        //2.设置Driver类对象
        job.setJarByClass(SaleSortDriver.class);
        //3.设置mapper和reducer类对象
        job.setMapperClass(SaleSortMapper.class);
        job.setReducerClass(SaleSortReducer.class);
        //4.设置map阶段输出的kv对象
        job.setMapOutputKeyClass(UserSaleSort.class);
        job.setMapOutputValueClass(Text.class);
        //5.设置最终输出的kv类对象
        job.setOutputKeyClass(Text.class);
        job.setOutputValueClass(UserSaleSort.class);
        //6.设置读取文件的路径 和 输出文件的路径
        FileInputFormat.setInputPaths(job,new Path("D:\\code\\sale_details (1).txt"));
        FileOutputFormat.setOutputPath(job,new Path("D:\\code\\output_compare"));
        //7.提交job
        boolean result = job.waitForCompletion(true);
        System.out.println(result?"计算成功":"计算失败");
    }

}

 结果