本地idea提交到yarn上运行错误解析

1.
Exception in thread "main" org.apache.hadoop.security.AccessControlException: Permission denied: user=yoohhwz, access=EXECUTE, inode="/tmp/hadoop-yarn":hadoop:supergroup:drwx------
解决方案:在main方法首行加上  System.setProperty("HADOOP_USER_NAME","hadoop"); 

2.
Exception message: /bin/bash: line 0: fg: no job control

Stack trace: ExitCodeException exitCode=1: /bin/bash: line 0: fg: no job control

 at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
 at org.apache.hadoop.util.Shell.run(Shell.java:455)
 at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
 at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211)
 at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
 at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
 at java.util.concurrent.FutureTask.run(FutureTask.java:262)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:745)
Container exited with a non-zero exit code 1

解决方案:
在resources下的mapred-site.xml,添加如下属性:

<property> 
       <name>mapred.remote.os</name>
        <value>hadoop001</value>
        <descript>Remote MapReduce framework's OS, can be either Linux or Windows</descript>
    </property>

    <property>
      <name>mapreduce.app-submission.cross-platform</name>
      <value>true</value>
    </property>

3.
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class com.hwz.bigdata.HDFS.WC2Yarn$MyMapper not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2199)
	at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:196)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:745)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
解决方案:
//将mvn packet后的包指定一下
conf.set("mapred.jar","C:\\tools\\IdealUltimateEdition\\hadoop\\target\\hadoop-1.0-SNAPSHOT.jar");
job.setJar("C:\\tools\\IdealUltimateEdition\\hadoop\\target\\hadoop-1.0-SNAPSHOT.jar");
  • 1
    点赞
  • 1
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值