方式1–查看作业记录文件
CDH中,在HDFS的”/user/history/done”目录下,包含了全部已完成的MR作业,”done_intermediate”包含了全部正在进行的作业。
在”/user/history/done”目录下,每个job有两个文件:job.jhist和job.xml,job.jhist是作业运行过程的详细记录,格式为json。job.xml是作业的配置文件,两者的示例结构如下。
job–.jhist文件结构
task信息(每行一个)
{"type":"TASK_STARTED","event":{"org.apache.hadoop.mapreduce.jobhistory.TaskStarted":{"taskid":"task_1465461051654_0001_r_000002","taskType":"REDUCE","startTime":1465461143745,"splitLocations":""}}}
counts(最后一行)
{"name":"DATA_LOCAL_MAPS","displayName":"Data-local map tasks","value":1},{"name":"SLOTS_MILLIS_MAPS","displayName":"Total time spent by all maps in occupied slot