自定义OutputFormat的使用方法

自定义OutputFormat使用

为了实现控制最终文件的输出路径和输出格式，可以自定义OutputFormat.

自定义OutFomat步骤
1)A定义类继承FileOutputFormat，返回B自定义类
2)B自定义类继承RecordWriter类,重写write()方法实现业务逻辑。

先继承FileOutFormat,返回FRecordWriter

public class FileterOutputFormat extends FileOutputFormat<Text, NullWritable>{	
	@Override
	public RecordWriter<Text, NullWritable> getRecordWriter(TaskAttemptContext job)
	throws IOException, InterruptedException {
		return new FRecordWriter(job);
	}
}

FRecordWriter继承RecordWriter类，在write()方法中实现具体业务逻辑

public class FRecordWriter extends RecordWriter<Text, NullWritable> {
	FSDataOutputStream fosatguigu;
	FSDataOutputStream fosother;
	public FRecordWriter(TaskAttemptContext job) {	
	try{	
			//1.获取文件系统
			FileSystem fs = FileSystem.get(job.getConfiguration());
			//2.创建输出到atgugui路径的输出流
			fosatguigu = fs.create(new Path("e:/atguigu.log"));
			//3.创建输出到other路径的输出流
			fosother = fs.create(new Path("e:/other.log"));
}catch(IOException e){
 	e.printStackTrace();
}
}

//关闭流
@Override
public void close(TaskAttemptContext context) throws Exception{
	IOUtils.closeStream(fosatguigu);
	IOUtils.closeStream(fosother);
}

//业务逻辑
public void write(Text key, NullWritable value) throws IOException, InterruptedException {
	if(key.toString.contains("atguigu")){
		fosatguigu.write(key.toString().getBytes());
	}else {
		fosother.write(key.toString().getBytes());
	}
}