hadoop-*-streaming.jar源码浅析

Main-Class为HadoopStreaming，主函数：

public static void main(String[] args) throws Exception {
    int returnStatus = 0;
    StreamJob job = new StreamJob();      
    returnStatus = ToolRunner.run(job, args);
    if (returnStatus != 0) {
      System.err.println("Streaming Job Failed!");
      System.exit(returnStatus);
    }
  }

核心类为StreamJob，StreamJob实现了org.apache.hadoop.util.Tool接口，其不带参数的构造函数做了两件事，调用setupOptions()初始化其支持的所有选项，以及初始化config_。

其支持的选项有：

name	desc	argName	max	required
input	DFS input file(s) for the map step	path	Integer.MAX_VALUE	true
output	DFS output directory for the reduce step	path	1	true
mapper	the streaming command to run	cmd	1	false
combiner	the streaming command to run	cmd	1	false
reducer	the streaming command to run	cmd	1	false
file	file to be shipped in the job jar file	file	Integer.MAX_VALUE	false
dfs	Optional,override DFS configu