关闭

Hadoop错误之 /bin/bash: /bin/java: No such file or directory

标签: hadoopjava软连接csrutilsip
236人阅读 评论(0) 收藏 举报
分类:

莫名的错误日志如下,该错误的产生原因大概可能由于重装hadoop环境有关

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/liuxunming/MyConfigure/hadoop-2.7.4/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/liuxunming/MyConfigure/apache-hive-2.3.1-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
18/01/05 14:17:28 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
18/01/05 14:17:29 INFO client.RMProxy: Connecting to ResourceManager at /127.0.0.1:8032
18/01/05 14:17:30 INFO input.FileInputFormat: Total input paths to process : 1
18/01/05 14:17:30 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library from the embedded binaries
18/01/05 14:17:30 INFO lzo.LzoCodec: Successfully loaded & initialized native-lzo library [hadoop-lzo rev f1deea9a313f4017dd5323cb8bbb3732c1aaccc5]
18/01/05 14:17:30 INFO mapreduce.JobSubmitter: number of splits:1
18/01/05 14:17:30 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1515042904174_0004
18/01/05 14:17:30 INFO impl.YarnClientImpl: Submitted application application_1515042904174_0004
18/01/05 14:17:30 INFO mapreduce.Job: The url to track the job: http://liuxunmingdeMacBook-Pro.local:8088/proxy/application_1515042904174_0004/
18/01/05 14:17:30 INFO mapreduce.Job: Running job: job_1515042904174_0004
18/01/05 14:17:33 INFO mapreduce.Job: Job job_1515042904174_0004 running in uber mode : false
18/01/05 14:17:33 INFO mapreduce.Job:  map 0% reduce 0%
18/01/05 14:17:33 INFO mapreduce.Job: Job job_1515042904174_0004 failed with state FAILED due to: Application application_1515042904174_0004 failed 2 times due to AM Container for appattempt_1515042904174_0004_000002 exited with  exitCode: 127
For more detailed output, check application tracking page:http://liuxunmingdeMacBook-Pro.local:8088/cluster/app/application_1515042904174_0004Then, click on links to logs of each attempt.
Diagnostics: Exception from container-launch.
Container id: container_1515042904174_0004_02_000001
Exit code: 127
Stack trace: ExitCodeException exitCode=127: 
    at org.apache.hadoop.util.Shell.runCommand(Shell.java:585)
    at org.apache.hadoop.util.Shell.run(Shell.java:482)
    at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:776)
    at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)


Container exited with a non-zero exit code 127
Failing this attempt. Failing the application.
18/01/05 14:17:33 INFO mapreduce.Job: Counters: 0

以上是错误日志,核心的内容有两句日志:

Stack trace: ExitCodeException exitCode=127
For more detailed output, check application tracking page:http://liuxunmingdeMacBook-Pro.local:8088/cluster/app/application_1515042904174_0004Then, click on links to logs of each attempt.

我们以第一句去网络上搜索相关答案,发现搜到的无外乎以下几种解决方案:
1,更改hadoop-env.sh中的java_home环境变量,如这里
2,更改yarn-site.xml中的yarn.application.classpath之类的classpath,如这里
3,添加mac中java软连接,如这里,以及链接4
试了以上几种方法后依然无法解决问题,然后在链接4的评论下又看到一篇文章,说能真正解决这个问题,因为以前遇到过这个问题,依稀记得当时是重启电脑输入什么命令后解决的,然后点进去链接看到文章里果然有我的评论,所以在此仅以此文记录下这个坑,以后再犯好找解决方法

以下是解决方法,参考这里
1,重启电脑,进入恢复模式,打开终端
2,输入以下命令,禁用SIP特性,SIP特性大概是指即使拥有sudo权限也无法修改系统级目录的权限,默认为启用状态

csrutil disable

3,重启电脑,进入正常模式,打开终端
在输入建立java软连接的命令

sudo ln -s /usr/bin/java /bin/java

输入系统密码就OK了,到此,此错误成功解决

后续:由于担心disable了csrutil会对系统本身有什么影响,于是打算再次开启,测试流程如下
先重启电脑,再运行job,第一次失败,提示什么safe mode,第二次成功
再重启电脑,进入恢复模式,启用csrutil

csrutil enable
successfully enabled system integrity protection,please restart the machine for the change to take effect 

再重启电脑,再运行job,第一次失败,提示什么safe mode,第二次成功
至此,得出hadoop环境已经正常。

0
0
查看评论

hadoop-2.0.6-alpha配置lzo

一、背景 最近使用bulkload往HBase中批量加载数据,正在进行优化工作,其中的一个优化点是使用lzo压缩,下面给出配置过程。 二、环境 2.1、软件环境 Red Hat 4.4.7-3 hadoop-2.0.6-alpha(此配置文档对hadoop-2.2.0同样适应) hbase-0.9...
  • wl101yjx
  • wl101yjx
  • 2014-12-05 22:02
  • 750

sqoop /user/local/jdk1.8.0_131/bin/java:No such file or directory.........

执行sqoop导数到hive的时候报错sqoop xxxx :UNsupported major.minor version 52.0,是jdk版本冲突(CDH默认安装的是jdk1.7),改掉环境变量之后(将jdk1.8环境变量删除,目录删除) 又报错:/user/local/jdk1.8.0_1...
  • u010469632
  • u010469632
  • 2017-11-22 16:41
  • 168

hadoop no such file or directory问题

在使用cygwin装完hadoop后,输入命令 ./hadoop version 出现如下错误 ./hadoop:line 297:c:\java\jdk1.6.0_05\bin/bin/java:No such file or directory ./hadoop:line 345...
  • snrqtdhuqf
  • snrqtdhuqf
  • 2011-11-17 14:24
  • 9163

JDK报错:-bash: /usr/local/java/bin/java: No such file or directory

在 Linux 系统中安装 JDK 环境,配置好环境变量后,输入 java、javac 或者 java -version 等时,都提示如下错误:-bash: /usr/local/java/bin/java: No such file or directory
  • tzhuwb
  • tzhuwb
  • 2017-09-03 08:28
  • 1178

spark yarn /bin/bash: /bin/java: is a directory

mac Os  10.12  +hadoop2.7.2+spark1.6.1  ./bin/spark-submit --class org.apache.spark.examples.SparkPi --master yarn --deploy-mo...
  • luyee2010
  • luyee2010
  • 2017-06-22 03:42
  • 359

Ubuntu安装jdk报错:-bash /usr/.../java:No such file or directroy

一大早在服务器上安装了个jdk,原来是干干净净的机器,但是装完jdk运行java -version后,输出 /usr/bin/java: No such file or directory;配置都没有问题,最后发现是jdk版本下错了,64位的机器装了个32位的jdk。 如下是解决办法:运...
  • blueskyliulan
  • blueskyliulan
  • 2015-09-16 12:00
  • 1849

关于解决line 1: #!/bin/bash: No such file or directory的问题

今天在执行脚本的时候,一直报这个错,虽然不影响脚本的执行,但是看见它的存在还是很不爽,然后就查了半天最终搞定 1.一般情况下,/bin下面不会不存在bash/,所以和平时我们遇到的No such file or directory并不一样 2.既然是执行的时候报错,个人感觉不是执行权限的问题 3.如...
  • biaoxianwokeng
  • biaoxianwokeng
  • 2016-09-16 16:21
  • 3766

ubuntu安装Java出现bash: ./java: No such file or directory的解决

今天在ubuntu系统上装jdk,下载、解压缩,配置环境变量后,用java -version测试,一直报错: # java -version -bash: /usr/local/jdk1.7.0_07/bin/java: 没有那个文件或目录 通过网络搜索,发现通过下面的方法可以解决这个错误。 ...
  • cao18995783056
  • cao18995783056
  • 2016-08-05 14:57
  • 2056

-bash: ./start.sh: /bin/sh^M: bad interpreter: No such file or directory 错误解决方案

问题描述:sh文件中,在win环境下,用WinSCP编辑,出现如下错误: -bash: ./start.sh: /bin/sh^M: bad interpreter: No such file or directory 解决方案:这是win的编码引起的,可通过如下解决。 1.查看该...
  • lizhengnanhua
  • lizhengnanhua
  • 2016-06-21 02:33
  • 3325

shell脚本执行错误:#!/bin/bash: No such file or directory

执行.sh脚本时控制台报错 : #!/bin/bash: No such file or directory解决办法:cat -A 文件路径会发现第一行有问题M-oM-;M-?#!/bin/bash$ #----------------------------------$ #$ # M-eM-...
  • yaoxiaofeng_000
  • yaoxiaofeng_000
  • 2016-10-11 17:34
  • 4481
    个人资料
    • 访问:176875次
    • 积分:2795
    • 等级:
    • 排名:第15138名
    • 原创:58篇
    • 转载:5篇
    • 译文:0篇
    • 评论:52条
    联系我吧
    我的App
    博客专栏
    文章分类