1.为何 hadoop streaming 命令 。bin/hadoop jar hadoop-streaming-1.1.2.jar -input sjtu -output shuchu -mapper cat -reducer wc 可以成功运行
bin/hadoop jar hadoop-streaming-1.1.2.jar -input sjtu -output shuchu -mapper cat -reducer wc -l (有空格) 却会报错
bin/hadoop jar hadoop-streaming-1.1.2.jar -input sjtu -output shuchu -mapper cat -reducer wc -l (无空格) 运行一段时间才会报错
自己编写了两个脚本 sh1.sh (cat) sh2.sh(wc ) 命令 bin/hadoop jar hadoop-streaming-1.1.2.jar -input sjtu -output sh -mapper /home/wangmeng/sh1.sh -reducer /home/wangmeng/sh2.sh 却会在最后报错
13/07/21 22:40:42 INFO streaming.StreamJob: map 0% reduce 0%
13/07/21 22:41:12 INFO streaming.StreamJob: map 100% reduce 100%
13/07/21 22:41:12 INFO streaming.StreamJob: To kill this job, run:
13/07/21 22:41:12 INFO streaming.StreamJob: /home/wangmeng/hadoop-1.1.2/libexec/../bin/hadoop job -Dmapred.job.tracker=master:9001 -kill job_201307212156_0013
13/07/21 22:41:12 INFO streaming.StreamJob: Tracking URL: http://master:50030/jobdetails.jsp?jobid=job_201307212156_0013
13/07/21 22:41:12 ERROR streaming.StreamJob: Job not successful. Error: # of failed Map Tasks exceeded allowed limit. FailedCount: 1. LastFailedTask: task_201307212156_0013_m_000000
13/07/21 22:41:12 INFO streaming.StreamJob: killJob...
Streaming Command Failed!