运行hadoop 报错 No job jar file set. User classes may not be found. See Job or Job#setJar(String)

如下创建主类程序

public class JobMain extends Configured implements Tool {
@Override
public int run(String[] args) throws Exception {
//1:创建job对象
Job job = Job.getInstance(super.getConf(), "mapreduce_sort");
...
}

结果单机上跑没有问题,但是放在集群上就会出问题
报如下错误

20/12/19 19:03:26 INFO client.RMProxy: Connecting to ResourceManager at node01/192.168.177.101:8032
20/12/19 19:03:26 WARN mapreduce.JobResourceUploader: No job jar file set.  User classes may not be found. See Job or Job#setJar(String).
20/12/19 19:03:26 INFO input.FileInputFormat: Total input paths to process : 1
20/12/19 19:03:30 INFO mapreduce.JobSubmitter: number of splits:1
20/12/19 19:03:30 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1608426890333_0007
20/12/19 19:03:30 INFO mapred.YARNRunner: Job jar is not present. Not adding any jar to the list of resources.
20/12/19 19:03:30 INFO impl.YarnClientImpl: Submitted application application_1608426890333_0007
20/12/19 19:03:30 INFO mapreduce.Job: The url to track the job: http://node01:8088/proxy/application_1608426890333_0007/
20/12/19 19:03:30 INFO mapreduce.Job: Running job: job_1608426890333_0007
20/12/19 19:04:10 INFO mapreduce.Job: Job job_1608426890333_0007 running in uber mode : true
20/12/19 19:04:10 INFO mapreduce.Job:  map 100% reduce 0%
20/12/19 19:04:11 INFO mapreduce.Job:  map 100% reduce 100%
20/12/19 19:04:12 INFO mapreduce.Job: Job job_1608426890333_0007 failed with state FAILED due to: Task failed task_1608426890333_0007_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0

20/12/19 19:04:12 INFO mapreduce.Job: Counters: 17
        Job Counters 
                Failed map tasks=1
                Failed reduce tasks=1
                Launched map tasks=1
                Launched reduce tasks=1
                Other local map tasks=1
                Total time spent by all maps in occupied slots (ms)=1065
                Total time spent by all reduces in occupied slots (ms)=155
                TOTAL_LAUNCHED_UBERTASKS=2
                NUM_UBER_SUBMAPS=1
                NUM_UBER_SUBREDUCES=1
                NUM_FAILED_UBERTASKS=2
                Total time spent by all map tasks (ms)=1065
                Total time spent by all reduce tasks (ms)=155
                Total vcore-milliseconds taken by all map tasks=1065
                Total vcore-milliseconds taken by all reduce tasks=155
                Total megabyte-milliseconds taken by all map tasks=1090560
                Total megabyte-milliseconds taken by all reduce tasks=158720

查阅了一下资料,原因是找不到job任务运行的资源,如map类、reduce类等。

public class JobMain extends Configured implements Tool {

    @Override
    public int run(String[] args) throws Exception {
        //1:创建job对象
        Job job = Job.getInstance(super.getConf(), "mapreduce_sort");
        //只要是Initmain类所在包下的任一个类名都可以,默认为当前 job所在类名
        job.setJarByClass(JobMain.class);
        ...

再回到集群上运行就好了。

  • 0
    点赞
  • 1
    收藏
    觉得还不错? 一键收藏
  • 1
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 1
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值