下面简单介绍MRUnit的使用方法
1、下载
http://archive.apache.org/dist/mrunit/mrunit-1.0.0/
本人下载:apache-mrunit-1.0.0-hadoop2-bin.tar.gz
2、安装
tar -axvf apache-mrunit-1.0.0-hadoop2-bin.tar.gz -C 指定的路径
将lib下除了commons-logging-1.1.1.jar,其余的jar都导入到测试工程中
3、测试
本例使用简化的MaxTemperature实例
Map类:
public class MaxTemperatureMapper extends Mapper<LongWritable, Text, Text, IntWritable> {
private static final int MISSING = 9999;
public void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException {
String line = value.toString();
String year = line.substring(0, 4);
int airTemperature;
airTemperature = Integer.parseInt(line.substring(5, 7));
String quality = line.substring(8, 11);
if (airTemperature != MISSING && quality.equals("123"))
context.write(new Text(year), new IntWritable(airTemperature));
}
}
Reduce类:
public class MaxTemperatureReducer extends Reducer<Text, IntWritable, Text, IntWritable> {
public void reduce(Text key, Iterable<IntWritable> values, Context context)
throws IOException, InterruptedException {
int maxValue = Integer.MIN_VALUE;
for (IntWritable val : values)
maxValue = Math.max(maxValue, val.get());
context.write(key, new IntWritable(maxValue));
}
}
Map测试类:
public class MaxTemperatureMapperTest {
@Test
public void processesValidRecord() throws IOException, InterruptedException {
Text value = new Text("1950-30-123"); //模拟的输入数据
int result = 30;
new MapDriver<LongWritable, Text, Text, IntWritable>()
.withMapper(new MaxTemperatureMapper()) //map类
.withInput(new LongWritable(0), value) //输入Text
.withOutput(new Text("1950"), new IntWritable(result)) //预期的输出结果
.runTest(); //运行测试
}
}
Reduce测试类:
public class MaxTemperatureReducerTest {
@Test
public void returnsMaximumIntegerInValues() throws IOException, InterruptedException {
new ReduceDriver<Text, IntWritable, Text, IntWritable>()
.withReducer(new MaxTemperatureReducer()) //Reduce类
.withInput(new Text("1950"), Arrays.asList(new IntWritable(10), new IntWritable(5))) //模拟的Reduce输入
.withOutput(new Text("1950"), new IntWritable(10)) //预期的输出结果
.runTest(); //运行测试
}
}
以jUnit test 方式运行测试类即可