Hadoop Pipes “Server failed to authenticate”错误及解决

在执行《hadoop实战》(第2版)3.5节的Hadoop Pipes示例时遇到'Server failed to authenticate'错误。通过检查makefile、创建目录、上传可执行文件并尝试运行,发现需要更新Hadoop的静态库并重启服务。按照相关链接提供的步骤,生成新的libhadooppipes.a和libhadooputils.a静态库,覆盖到相应目录,并重启Hadoop后,程序成功运行。
摘要由CSDN通过智能技术生成

问题描述:

《hadoop实战》(第2版)3.5节的Hadoop Pipes例子。

makefile的内容:
HADOOP_INSTALL=/home/xxl/hadoop-1.1.2
PLATFORM=Linux-i386-32
SSL_INSTALL=/usr/local/ssl
CC=g++
CPPFLAGS=-m32 -I$(HADOOP_INSTALL)/c++/$(PLATFORM)/include -I$(SSL_INSTALL)/include
wordcount: wordcount.cpp
    $(CC) $(CPPFLAGS) $< -Wall -L$(HADOOP_INSTALL)/c++/$(PLATFORM)/lib -lhadooppipes -lhadooputils \
        -L$(SSL_INSTALL)/lib -lcrypto -lssl -ldl -lpthread -g -O2 -o $@

上述makefile内容没有问题

将可执行文件上传到bin文件夹内
~/hadoop-1.1.2/bin/hadoop fs -mkdir bin
~/hadoop-1.1.2/bin/hadoop dfs -put wordcount bin

运行这个wordcount程序
~/hadoop-1.1.2/bin/hadoop pipes -D hadoop.pipes.java.recordreader=true -D hadoop.pipes.java.recordwriter=true -input input -output output -program bin/wordcount


此时出现如下问题:

xxl@xxl-pc:~/MapReduce/wordcount_cpp$ ~/hadoop-1.1.2/bin/hadoop pipes -D hadoop.pipes.java.recordreader=true -D hadoop.pipes.java.recordwriter=true -input /user/xxl/input/file0* -output /user/xxl/output/outputfile -program bin/wordcount
13/10/04 22:29:21 WARN mapred.JobClient: No job jar file set.  User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
13/10/04 22:29:22 INFO util.NativeCodeLoader: Loaded the native-hadoop library
13/10/04 22:29:22 WARN snappy.LoadSnappy: Snappy native library not loaded
13/10/04 22:29:22 INFO mapred.FileInputFormat: Total input paths to process : 2
13/10/04 22:29:22 INFO mapred.JobClient: Running job: job_201310041509_0017
13/10/04 22:29:23 INFO mapred.JobClient:  map 0% reduce 0%
13/10/04 22:29:32 INFO mapred.JobClient: Task Id : attempt_201310041509_0017_m_000000_0, Status : FAILED
java.io.IOException
	at org.apache.hadoop.mapred.pipes.OutputHandler.waitForAuthentication(OutputHandler.java:188)
	at org.apache.hadoop.mapred.pipes.Application.waitForAuthentication(Application.java:194)
	at org.apache.hadoop.mapred.pipes.Application.<init>(Application.java:149)
	at org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:68)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
	at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:416)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
	at org.apache.hadoop.mapred.Child.main(Child.java:249)

attempt_201310041509_0017_m_00000
  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值