Hadoop权威指南---第二章MaxTemperature例题源码

最新推荐文章于 2019-01-10 16:32:58 发布

acm_lkl

最新推荐文章于 2019-01-10 16:32:58 发布

阅读量900

点赞数 1

分类专栏： hadoop学习文章标签： hadoop mapreduce

本文链接：https://blog.csdn.net/acm_lkl/article/details/41399363

版权

hadoop学习专栏收录该内容

1 篇文章 0 订阅

订阅专栏

敲了一下hadoop权威指南第二章的例题，虽然基本上是照着书上敲的，但还是把它放到这方便以后查看。

代码如下：

<span style="font-size:18px;"><span style="font-size:18px;">import java.io.IOException;
import java.util.StringTokenizer;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.Mapper;
import org.apache.hadoop.mapreduce.Reducer;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
import org.apache.hadoop.util.GenericOptionsParser;

public class MaxTemperature {

	  static class MaxTemperatureMapper
	        extends Mapper<LongWritable, Text, Text ,IntWritable >{
		    private static final int MISSING = 9999;
		    public void map(LongWritable key,Text value, Context context)
		            throws IOException, InterruptedException{
		    	String line = value.toString();
		    	String year = line.substring(15, 19);
		    	int airTemperature ;
		    	if (line.charAt(29) == '+'){
		    		airTemperature  = Integer.parseInt(line.substring(30, 35));
		    	}
		    	else{
		    		airTemperature = Integer.parseInt(line.substring(29, 35));
		    	}
		    	String quality = line.substring(35, 36);
		    	if(airTemperature != MISSING && quality.matches("[01459]"));{
		    		context.write(new Text(year), new IntWritable(airTemperature));
		    	}
		    }
	  }
	  
	  
	  static class MaxTemperatureReducer 
	     extends Reducer <Text , IntWritable, Text, IntWritable>{
		 public void reduce (Text key, Iterable<IntWritable> values,
				 Context context)
				 throws IOException, InterruptedException{
			 
			 int maxValue =  Integer.MIN_VALUE;
			 for (IntWritable value : values){
				 maxValue=Math.max(maxValue, value.get());
			 }
			 context.write(key, new IntWritable(maxValue));
		 }
	}
	  
	public static void main(String[] args) throws IOException, ClassNotFoundException, InterruptedException{
		if (args.length !=2){
			System.err.println("Usage: MaxTemperature <input path> <output path>");
			System.exit(-1);
		}
	    
		Job  job =new Job();
		job.setJarByClass(MaxTemperature.class);
		
		FileInputFormat.addInputPath(job, new Path(args[0]));
		FileOutputFormat.setOutputPath(job,new Path(args[1]));
		
		job.setMapperClass(MaxTemperatureMapper.class);
		job.setReducerClass(MaxTemperatureReducer.class);
		job.setOutputKeyClass(Text.class);
		job.setOutputValueClass(IntWritable.class);
		
		System.exit(job.waitForCompletion(true) ? 0 : 1);
	}

}
</span></span>

输出结果如下（输入数据是自己随便造的，输出的所有气温值都乘了10）