Hadoop Error: java.io.IOException: Unable to initialize any output collector

[hadoop@hadoop01 ~]$ hadoop jar mr.jar cn.hadoop.mr.WCRunner
16/07/24 16:52:08 INFO client.RMProxy: Connecting to ResourceManager at hadoop01/192.168.56.200:8032
16/07/24 16:52:09 WARN mapreduce.JobSubmitter: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
16/07/24 16:52:09 INFO input.FileInputFormat: Total input paths to process : 1
16/07/24 16:52:09 INFO mapreduce.JobSubmitter: number of splits:1
16/07/24 16:52:09 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1469343376698_0001
16/07/24 16:52:10 INFO impl.YarnClientImpl: Submitted application application_1469343376698_0001
16/07/24 16:52:10 INFO mapreduce.Job: The url to track the job: http://hadoop01:8088/proxy/application_1469343376698_0001/
16/07/24 16:52:10 INFO mapreduce.Job: Running job: job_1469343376698_0001
16/07/24 16:52:20 INFO mapreduce.Job: Job job_1469343376698_0001 running in uber mode : false
16/07/24 16:52:20 INFO mapreduce.Job:  map 0% reduce 0%
16/07/24 16:52:25 INFO mapreduce.Job: Task Id : attempt_1469343376698_0001_m_000000_0, Status : FAILED
Error: java.io.IOException: Unable to initialize any output collector
	at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:412)
	at org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:81)
	at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:695)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:767)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)

16/07/24 16:52:30 INFO mapreduce.Job: Task Id : attempt_1469343376698_0001_m_000000_1, Status : FAILED
Error: java.io.IOException: Unable to initialize any output collector
	at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:412)
	at org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:81)
	at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:695)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:767)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)

16/07/24 16:52:35 INFO mapreduce.Job: Task Id : attempt_1469343376698_0001_m_000000_2, Status : FAILED
Error: java.io.IOException: Unable to initialize any output collector
	at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:412)
	at org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:81)
	at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:695)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:767)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)

16/07/24 16:52:42 INFO mapreduce.Job:  map 100% reduce 100%
16/07/24 16:52:42 INFO mapreduce.Job: Job job_1469343376698_0001 failed with state FAILED due to: Task failed task_1469343376698_0001_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0

16/07/24 16:52:42 INFO mapreduce.Job: Counters: 9
	Job Counters 
		Failed map tasks=4
		Launched map tasks=4
		Other local map tasks=3
		Data-local map tasks=1
		Total time spent by all maps in occupied slots (ms)=15512
		Total time spent by all reduces in occupied slots (ms)=0
		Total time spent by all map tasks (ms)=15512
		Total vcore-seconds taken by all map tasks=15512
		Total megabyte-seconds taken by all map tasks=15884288


错误原因如下:

public class WCRunner {
    public static void main(String[] args) throws IOException, ClassNotFoundException, InterruptedException {
        
        Configuration conf = new Configuration();
        Job wcjob = Job.getInstance(conf);
        
        wcjob.setJarByClass(WCRunner.class);
        
        wcjob.setMapperClass(WCMapper.class);
        wcjob.setReducerClass(WCReducer.class);
        
        wcjob.setOutputKeyClass(Text.class);
        wcjob.setOutputValueClass(LongWritable.class);
        
        wcjob.setMapOutputKeyClass(Text.class);
        wcjob.setMapOutputValueClass(LongWritable.class);
        
        FileInputFormat.setInputPaths(wcjob, "/wc/inputdata/");
        FileOutputFormat.setOutputPath(wcjob, new Path("/wc/output/"));
        
        wcjob.waitForCompletion(true);
    }
}

这里的Text是import com.sun.jersey.core.impl.provider.entity.XMLJAXBElementProvider.Text;

正确的类型是import org.apache.hadoop.io.Text;

评论 2
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值