当然可以。为了诊断问题,我们可以在Mapper和Reducer的关键部分添加一些打印语句来确认流程和输出。这样,当你运行程序时,可以在Hadoop的日志输出中看到这些打印语句。
下面是修改后的部分代码:
1. **Mapper**:
```java
protected void map(LongWritable key, Text value, Mapper<LongWritable, Text, Text, Text>.Context context)
throws IOException, InterruptedException {
String line = value.toString();
String[] values = line.split(" ");
System.out.println("Mapper Input Line: " + line);
context.write(new Text(values[0]), new Text(values[1]+"0"));
System.out.println("Mapper Output Key-Value: " + values[0] + " -> " + values[1]+"0");
context.write(new Text(values[1]), new Text(values[0]+"1"));
System.out.println("Mapper Output Key-Value: " + values[1] + " -> " + values[0]+"1");
}
```
2. **Reducer**:
```java
protected void reduce(Text key, Iterable<Text> values, Reducer<Text, Text, Text, Text>.Context context)
throws IOException, InterruptedException {
List<String> grand = new ArrayList<String>();
List<String> child = new ArrayList<String>();
String a = "1";
String b = "1";
System.out.println("Reducer Input Key: " + key.toString());
for(Text val:values){
System.out.println("Reducer Input Value: " + val.toString());
if(val.toString().endsWith("0")){
grand.add(val.toString());
a = "2";
}else if(val.toString().endsWith("1")){
child.add(val.toString());
b = "2";
}
}
if(a.equals("2") && b.equals("2")) {
for(String grap:grand){
for(String chis:child){
context.write(new Text(chis.substring(0, chis.length()-1)), new Text(grap.substring(0, grap.length()-1)));
System.out.println("Reducer Output Key-Value: " + chis.substring(0, chis.length()-1) + " -> " + grap.substring(0, grap.length()-1));
}
}
}
}
```
这些打印语句会帮助你了解Mapper和Reducer在处理输入数据时的具体步骤和输出。
不过,请注意,由于Hadoop是一个分布式框架,这些打印信息可能会散布在不同的日志文件中,具体取决于哪个节点正在执行该任务。所以,当你检查日志时,确保查看所有相关的节点日志。
023-10-26 18:27:53,844 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2023-10-26 18:27:53,943 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false
Mapper Input Line: child parent
Mapper Output Key-Value: child -> 0
Mapper Output Key-Value: -> child1
Mapper Input Line: Steven Lucy
Mapper Output Key-Value: Steven -> 0
Mapper Output Key-Value: -> Steven1
Mapper Input Line: Steven Jack
Mapper Output Key-Value: Steven -> 0
Mapper Output Key-Value: -> Steven1
Mapper Input Line: Jone Lucy
Mapper Output Key-Value: Jone -> 0
Mapper Output Key-Value: -> Jone1
Mapper Input Line: Jone Jack
Mapper Output Key-Value: Jone -> 0
Mapper Output Key-Value: -> Jone1
Mapper Input Line: Lucy Mary
Mapper Output Key-Value: Lucy -> 0
Mapper Output Key-Value: -> Lucy1
Mapper Input Line: Lucy Frank
Mapper Output Key-Value: Lucy -> 0
Mapper Output Key-Value: -> Lucy1
Mapper Input Line: Jack Alice
Mapper Output Key-Value: Jack -> 0
Mapper Output Key-Value: -> Jack1
Mapper Input Line: Jack Jesse
Mapper Output Key-Value: Jack -> 0
Mapper Output Key-Value: -> Jack1
Mapper Input Line: David Alice
Mapper Output Key-Value: David -> 0
Mapper Output Key-Value: -> David1
Mapper Input Line: David Jesse
Mapper Output Key-Value: David -> 0
Mapper Output Key-Value: -> David1
Mapper Input Line: Philip David
Mapper Output Key-Value: Philip -> 0
Mapper Output Key-Value: -> Philip1
Mapper Input Line: Philip Alma
Mapper Output Key-Value: Philip -> 0
Mapper Output Key-Value: -> Philip1
Mapper Input Line: Mark David
Mapper Output Key-Value: Mark -> 0
Mapper Output Key-Value: -> Mark1
Mapper Input Line: Mark Alma
Mapper Output Key-Value: Mark -> 0
Mapper Output Key-Value: -> Mark1
2023-10-26 18:27:54,258 INFO mapred.LocalJobRunner:
2023-10-26 18:27:54,263 INFO mapred.MapTask: Starting flush of map output
2023-10-26 18:27:54,263 INFO mapred.MapTask: Spilling map output
2023-10-26 18:27:54,263 INFO mapred.MapTask: bufstart = 0; bufend = 232; bufvoid = 104857600
Reducer Input Key:
Reducer Input Value: Jone1
Reducer Input Value: Lucy1
Reducer Input Value: Mark1
Reducer Input Value: Steven1
Reducer Input Value: Philip1
Reducer Input Value: Steven1
Reducer Input Value: Philip1
Reducer Input Value: Jone1
Reducer Input Value: David1
Reducer Input Value: David1
Reducer Input Value: Jack1
Reducer Input Value: child1
Reducer Input Value: Jack1
Reducer Input Value: Mark1
Reducer Input Value: Lucy1
Reducer Input Key: David
Reducer Input Value: 0
Reducer Input Value: 0
Reducer Input Key: Jack
Reducer Input Value: 0
Reducer Input Value: 0
Reducer Input Key: Jone
Reducer Input Value: 0
Reducer Input Value: 0
Reducer Input Key: Lucy
Reducer Input Value: 0
Reducer Input Value: 0
Reducer Input Key: Mark
Reducer Input Value: 0
Reducer Input Value: 0
Reducer Input Key: Philip
Reducer Input Value: 0
Reducer Input Value: 0
Reducer Input Key: Steven
Reducer Input Value: 0
Reducer Input Value: 0
Reducer Input Key: child
Reducer Input Value: 0
if (values.length < 2) { System.err.println("Invalid line: " + line); return; }
if (values[0].trim().isEmpty() || values[1].trim().isEmpty()) { System.err.println("Empty value detected in line: " + line); return; }