1.
Exception in thread "main" org.apache.hadoop.security.AccessControlException: Permission denied: user=yoohhwz, access=EXECUTE, inode="/tmp/hadoop-yarn":hadoop:supergroup:drwx------
解决方案:在main方法首行加上 System.setProperty("HADOOP_USER_NAME","hadoop");
2.
Exception message: /bin/bash: line 0: fg: no job control
Stack trace: ExitCodeException exitCode=1: /bin/bash: line 0: fg: no job control
at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
at org.apache.hadoop.util.Shell.run(Shell.java:455)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Container exited with a non-zero exit code 1
解决方案:
在resources下的mapred-site.xml,添加如下属性:
<property>
<name>mapred.remote.os</name>
<value>hadoop001</value>
<descript>Remote MapReduce framework's OS, can be either Linux or Windows</descript>
</property>
<property>
<name>mapreduce.app-submission.cross-platform</name>
<value>true</value>
</property>
3.
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class com.hwz.bigdata.HDFS.WC2Yarn$MyMapper not found
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2199)
at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:196)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:745)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
解决方案:
//将mvn packet后的包指定一下
conf.set("mapred.jar","C:\\tools\\IdealUltimateEdition\\hadoop\\target\\hadoop-1.0-SNAPSHOT.jar");
job.setJar("C:\\tools\\IdealUltimateEdition\\hadoop\\target\\hadoop-1.0-SNAPSHOT.jar");
本地idea提交到yarn上运行错误解析
最新推荐文章于 2024-06-02 16:03:23 发布