Error: java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 1

最近搭建hadoop遇到如下问题,如何解决呢?

map.py 文件如下:

 reducer.py文件如下:

执行命令 :hadoop jar /opt/hadoop/hadoop/share/hadoop/tools/lib/hadoop-streaming-2.7.5.jar -file map.py -mapper "python3 map.py" -file reducer.py -reducer "python3 reducer.py" -input /word.txt -output /test9

报错如下怎么解决呢:

23/08/14 16:38:26 INFO client.RMProxy: Connecting to ResourceManager at /10.17.49.65:18040
23/08/14 16:38:27 INFO client.RMProxy: Connecting to ResourceManager at /10.17.49.65:18040
23/08/14 16:38:28 INFO mapred.FileInputFormat: Total input paths to process : 1
23/08/14 16:38:29 INFO mapreduce.JobSubmitter: number of splits:2
23/08/14 16:38:29 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1690419827373_0010
23/08/14 16:38:29 INFO impl.YarnClientImpl: Submitted application application_1690419827373_0010
23/08/14 16:38:29 INFO mapreduce.Job: The url to track the job: http://caihonlinetst01:18088/proxy/application_1690419827373_0010/
23/08/14 16:38:29 INFO mapreduce.Job: Running job: job_1690419827373_0010
23/08/14 16:38:44 INFO mapreduce.Job: Job job_1690419827373_0010 running in uber mode : false
23/08/14 16:38:44 INFO mapreduce.Job:  map 0% reduce 0%
23/08/14 16:38:59 INFO mapreduce.Job:  map 50% reduce 0%
23/08/14 16:39:00 INFO mapreduce.Job:  map 100% reduce 0%
23/08/14 16:39:09 INFO mapreduce.Job: Task Id : attempt_1690419827373_0010_r_000000_0, Status : FAILED
Error: java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 1
    at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:322)
    at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:535)
    at org.apache.hadoop.streaming.PipeReducer.close(PipeReducer.java:134)
    at org.apache.hadoop.io.IOUtils.cleanup(IOUtils.java:245)
    at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:459)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1754)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
 

 

 

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值