Driver
思路:
注释已经标好
这个类需要改的地方
1.` 输入输出的地址
Path inpath=new Path("D:\\About_study\\idea\\workplace\\input\\weatherdata.txt");
Path outpath=new Path("D:\\About_study\\idea\\workplace\\output2");
2`
Map和Reducer的key和value的类型 主义和前面所对应,
//4设置mapper阶段输出的key和value类型
job.setMapOutputKeyClass(Text.class);
job.setMapOutputValueClass(NullWritable.class);
//5设置最终数据输出的key和value类型
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(NullWritable.class);
整体完整代码
Configuration conf = new Configuration();
//1 获取job对象
Job job = Job.getInstance(conf);
//2设置jar存储位置
job.setJarByClass(WeatherDriver.class);
//3关联map和reduce
job.setMapperClass(WeatherMap.class);
job.setReducerClass(WeatherReducer.class);
//4设置mapper阶段输出的key和value类型
job.setMapOutputKeyClass(Text.class);
job.setMapOutputValueClass(NullWritable.class);
//5设置最终数据输出的key和value类型
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(NullWritable.class);
//6设置输入路径和输出路径
//FileInputFormat.setInputPaths(job, new Path("D:\\About_study\\idea\\workplace\\input\\hellow.txt"));
// FileOutputFormat.setOutputPath(job, new Path("D:\\About_study\\idea\\workplace\\output"));
Path p=setPath(job);
p.getFileSystem(conf).delete(p,true);
//7提交job
Boolean result = job.waitForCompletion(true);
if (result == true) {
System.out.println("success");
} else {
System.out.println("failed");
}
System.exit(result ? 0 : 1);
}
private static Path setPath(Job job) throws IOException {
Path inpath=new Path("D:\\About_study\\idea\\workplace\\input\\weatherdata.txt");
Path outpath=new Path("D:\\About_study\\idea\\workplace\\output2");
FileInputFormat.addInputPath(job,inpath);
FileOutputFormat.setOutputPath(job,outpath);
return outpath;
}