mapreduce程序执行问题汇总

1.java.lang.NullPointerException     at org.apache.hadoop.io.WritableComparator.compare

如果extends WritableComparator 实现自定义的分组器

在无参构造中一定要添加super：

public NaturalKeyGroupingComparator() {
    /**
     * 一定要写 super(CompositeKey.class, true); 否则报下面的错 Error:
     * java.lang.NullPointerException at
     * org.apache.hadoop.io.WritableComparator.compare(WritableComparator.java:157)
     * 
     */
    super(CompositeKey.class, true);
}

2.java.lang.RuntimeException: java.io.EOFException     at org.apache.hadoop.io.WritableComparator.compare

这是序列化反序列化顺序不一致导致：

原代码：

    @Override
    public void write(DataOutput out) throws IOException {
        out.write(year);
        out.write(month);
        out.write(day);
        out.write(wd);
    }

    /**
     * 反序列化
     * @param in
     * @throws IOException
     */
    @Override
    public void readFields(DataInput in) throws IOException {
        this.year = in.readInt();
        this.month = in.readInt();
        this.day = in.readInt();
        this.wd = in.readInt();
    }

修改后：

    /**
     * 序列化输出
     * @param out
     * @throws IOException
     */
    @Override
    public void write(DataOutput out) throws IOException {
        out.writeInt(year);
        out.writeInt(month);
        out.writeInt(day);
        out.writeInt(wd);
    }

    /**
     * 反序列化
     * @param in
     * @throws IOException
     */
    @Override
    public void readFields(DataInput in) throws IOException {
        this.year = in.readInt();
        this.month = in.readInt();
        this.day = in.readInt();
        this.wd = in.readInt();
    }