Hadoop中MultipleOutputs的应用,以及自定义Writable

MultipleOutputs的应用

①将结果生成至不同文件夹

private MultipleOutputs<Text, Text> outputs=null;
@Override
protected void setup(Context context) throws Exception {
    // TODO Auto-generated method stub
    outputs=new MultipleOutputs<Text, Text>(context);
}

String phoneNum=key.toString();
String baseOutputPath=phoneNum.substring(0, 3)+"/"+phoneNum.substring(3, 6)+"/";
Text value 
= new Text("上行:"+upTotal+"\t下行:"+downTotal+"\t总流量:"+(upTotal+downTotal));
outputs.write(key, value, baseOutputPath);

②通过不同形式生成结果

//命名说输出(JobSubmit)
MultipleOutputs.
addNamedOutput(job, "text", TextOutputFormat.class, Text.class, Text.class);
MultipleOutputs.
addNamedOutput(job, "seq", SequenceFileOutputFormat.class, Text.class, Text.class);
MultipleOutputs.
addNamedOutput(job, "map", MapFileOutputFormat.class, Text.class, Text.class);
//Reducer
outputs.write("text", key, value, baseOutputPath+"text/");
outputs.write("seq", key, value, baseOutputPath+"seq/");
outputs.write("map", key, value, baseOutputPath+"map/");

自定义Writable

//注意实现WritableComparable接口
public class FlowWritable implements WritableComparable<FlowWritable> {
//定义参数以及提供get、set方法
    private int upflow;
    private int downflow;
    public int getUpflow() {
        return upflow;
    }
    public void setUpflow(int upflow) {
        this.upflow = upflow;
    }
    public int getDownflow() {
        return downflow;
    }
    public void setDownflow(int downflow) {
        this.downflow = downflow;
    }
//提供write方法
    public void write(DataOutput out) throws IOException {
        // TODO Auto-generated method stub
        out.writeInt(upflow);
        out.writeInt(downflow);
    }
//提供readField方法
    public void readFields(DataInput in) throws IOException {
        // TODO Auto-generated method stub
        this.upflow=in.readInt();
        this.downflow=in.readInt();
    }
//实现compareTo方法
    public int compareTo(FlowWritable o) {
        // TODO Auto-generated method stub
        return 0;
    }
//覆盖toString方法
    @Override
    public String toString() {
        return  upflow + " " + downflow;
    }
}
  • 0
    点赞
  • 1
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值