WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin

问题:

    每个人安装hadoop时,都可能会遇见这个问题“WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable“,然后转手就去百度。也不知道网上怎么传的,都说是版本问题,各种编译就上来了,我就日了,这坑了多少人,几天的时间就这么被坑了。


正解:

    作为程序员,遇见问题我们第一时间应该是去看日志的,伸手党们习惯直接百度,然后一步一步走弯路。


    由于hadoop默认的日志级别较低,执行命令export HADOOP_ROOT_LOGGER=DEBUG,console改一下,会发现日志里面写的很清晰:


17/11/06 18:53:16 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of successful kerberos logins and latency (milliseconds)])
17/11/06 18:53:16 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of failed kerberos logins and latency (milliseconds)])
17/11/06 18:53:16 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[GetGroups])
17/11/06 18:53:16 DEBUG impl.MetricsSystemImpl: UgiMetrics, User and group related metrics
17/11/06 18:53:16 DEBUG security.Groups:  Creating new Groups object
17/11/06 18:53:16 DEBUG util.NativeCodeLoader: Trying to load the custom-built native-hadoop library...
17/11/06 18:53:16 DEBUG util.NativeCodeLoader: Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: /hadoop/hadoop-2.7.1/lib/native/libhadoop.so.1.0.0: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /hadoop/hadoop-2.7.1/lib/native/libhadoop.so.1.0.0)
17/11/06 18:53:16 DEBUG util.NativeCodeLoader: java.library.path=/hadoop/hadoop-2.7.1/lib/native
17/11/06 18:53:16 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17/11/06 18:53:16 DEBUG util.PerformanceAdvisory: Falling back to shell based
17/11/06 18:53:16 DEBUG security.JniBasedUnixGroupsMappingWithFallback: Group mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
17/11/06 18:53:16 DEBUG security.Groups: Group mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; cacheTimeout=300000; warningDeltaMs=5000
17/11/06 18:53:17 DEBUG security.UserGroupInformation: hadoop login
17/11/06 18:53:17 DEBUG security.UserGroupInformation: hadoop login commit
17/11/06 18:53:17 DEBUG security.UserGroupInformation: using local user:UnixPrincipal: root
17/11/06 18:53:17 DEBUG security.UserGroupInformation: Using user: "UnixPrincipal: root" with name root
17/11/06 18:53:17 DEBUG security.UserGroupInformation: User entry: "root"
17/11/06 18:53:17 DEBUG security.UserGroupInformation: UGI loginUser:root (auth:SIMPLE)
17/11/06 18:53:17 DEBUG security.UserGroupInformation: PrivilegedAction as:root (auth:SIMPLE) from:org.apache.hadoop.hdfs.tools.GetConf.run(GetConf.java:315)


    一眼就能看出问题的所在,glibc问题,人家需要的版本高,你应该升级一下。

使用命令:strings /lib64/libc.so.6 |grep GLIBC_

[root@hadoop02 sbin]# strings /lib64/libc.so.6 |grep GLIBC_
GLIBC_2.2.5
GLIBC_2.2.6
GLIBC_2.3
GLIBC_2.3.2
GLIBC_2.3.3
GLIBC_2.3.4
GLIBC_2.4
GLIBC_2.5
GLIBC_2.6
GLIBC_2.7
GLIBC_2.8
GLIBC_2.9
GLIBC_2.10
GLIBC_2.11
GLIBC_2.12
GLIBC_PRIVATE


发现自己的版本低,解决办法就是升级,引用别人的劳动成果。

     http://bbotte.com/more-info/upgrade-glibc-version-to-solve-glibc_2-14-not-found/

     上一篇遇见问题,参考下面解决了。

    http://www.linuxdiyf.com/linux/23702.html



  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值