java使用job报错,java-链??接地图减少工作时出错

我的地图缩小结构

public class ChainingMapReduce {

public static class ChainingMapReduceMapper

extends Mapper{

public void map(Object key, Text value, Context context

) throws IOException, InterruptedException {

// code

}

}

}

public static class ChainingMapReduceReducer

extends Reducer {

public void reduce(Text key, Iterable values,

Context context

) throws IOException, InterruptedException {

//code

}

}

public static class ChainingMapReduceMapper1

extends Mapper{

public void map(Object key, Text value, Context context

) throws IOException, InterruptedException {

//code

}

}

}

public static class ChainingMapReduceReducer1

extends Reducer {

public void reduce(Text key, Iterable values,

Context context

) throws IOException, InterruptedException {

//code

}

}

public static void main(String[] args) throws IOException, InterruptedException, ClassNotFoundException {

Configuration conf = new Configuration();

Job job = new Job(conf, "First");

job.setJarByClass(ChainingMapReduce.class);

job.setMapperClass(ChainingMapReduceMapper.class);

job.setCombinerClass(ChainingMapReduceReducer.class);

job.setReducerClass(ChainingMapReduceReducer.class);

job.setOutputKeyClass(Text.class);

job.setOutputValueClass(IntWritable.class);

FileInputFormat.addInputPath(job, new Path("/home/Desktop/log"));

FileOutputFormat.setOutputPath(job, new Path("/home/Desktop/temp/output"));

job.waitForCompletion( true );

System.out.println("First Job Completed.....Starting Second Job");

System.out.println(job.isSuccessful());

/* FileSystem hdfs = FileSystem.get(conf);

Path fromPath = new Path("/home/Desktop/temp/output/part-r-00000");

Path toPath = new Path("/home/Desktop/temp/output1");

hdfs.rename(fromPath, toPath);

conf.clear();

*/

if(job.isSuccessful()){

Configuration conf1 = new Configuration();

Job job1 = new Job(conf1,"Second");

job1.setJarByClass(ChainingMapReduce.class);

job1.setMapperClass(ChainingMapReduceMapper1.class);

job1.setCombinerClass(ChainingMapReduceReducer1.class);

job1.setReducerClass(ChainingMapReduceReducer1.class);

job1.setOutputKeyClass(Text.class);

job1.setOutputValueClass(IntWritable.class);

FileInputFormat.addInputPath(job, new Path("/home/Desktop/temp/output/part-r-00000)");

FileOutputFormat.setOutputPath(job, new Path("/home/Desktop/temp/output1"));

System.exit(job1.waitForCompletion(true) ? 0 : 1);

}

System.exit(job.waitForCompletion(true) ? 0 : 1);

}

}

当我运行该程序时…第一个作业得到了完美执行,随后出现以下错误:

First Job Completed…..Starting Second Job true

12/01/27 15:24:21 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics

with processName=JobTracker, sessionId= – already initialized 12/01/27

15:24:21 WARN mapred.JobClient: Use GenericOptionsParser for parsing

the arguments. Applications should implement Tool for the same.

12/01/27 15:24:21 WARN mapred.JobClient: No job jar file set. User

classes may not be found. See JobConf(Class) or

JobConf#setJar(String). 12/01/27 15:24:21 INFO mapred.JobClient:

Cleaning up the staging area

file:/tmp/hadoop/mapred/staging/4991311720439552/.staging/job_local_0002

Exception in thread “main”

org.apache.hadoop.mapred.InvalidJobConfException: Output directory not

set. at

org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:123)

at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:872) at

org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:833) at

java.security.AccessController.doPrivileged(Native Method) at

javax.security.auth.Subject.doAs(Subject.java:396) at

org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)

at

org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:833)

at org.apache.hadoop.mapreduce.Job.submit(Job.java:476) at

org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:506) at

ChainingMapReduce.main(ChainingMapReduce.java:129)

我尝试将“ conf”用于两个作业,将“ conf”“ conf1”用于各个作业.

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值