三、Unable to load native-hadoop library for your platform... using builtin-java classes where appl...

 hadoop安装完成后，找些教程想看看hdfs命令使用，发现不管用什么老报下面这个警告：

 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable。

1、查看日志信息，找寻报错原因；

$ export HADOOP_ROOT_LOGGER=DEBUG,console    # 输出日志

$ hadoop fs -ls /

显示日志信息：

 

18/04/29 21:38:44 DEBUG util.Shell: setsid exited with exit code 0

18/04/29 21:38:44 DEBUG conf.Configuration: parsing URL jar:file:/usr/local/hadoop-2.9.0/share/hadoop/common/hadoop-common-2.9.0.jar!/core-default.xml

18/04/29 21:38:44 DEBUG conf.Configuration: parsing input stream sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@54b24c03

18/04/29 21:38:44 DEBUG conf.Configuration: parsing URL file:/usr/local/hadoop-2.9.0/etc/hadoop/core-site.xml

18/04/29 21:38:44 DEBUG conf.Configuration: parsing input stream java.io.BufferedInputStream@7bc9a682

18/04/29 21:38:44 DEBUG core.Tracer: sampler.classes = ; loaded no samplers

18/04/29 21:38:44 DEBUG core.Tracer: span.receiver.classes = ; loaded no span receivers

18/04/29 21:38:45 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess with annotation @org.apache.hadoop.metrics2.annotation.Metric(value=[Rate of successful kerberos logins and latency (milliseconds)], about=, valueName=Time, type=DEFAULT, always=false, sampleName=Ops)

18/04/29 21:38:45 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure with annotation @org.apache.hadoop.metrics2.annotation.Metric(value=[Rate of failed kerberos logins and latency (milliseconds)], about=, valueName=Time, type=DEFAULT, always=false, sampleName=Ops)

18/04/29 21:38:45 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with annotation @org.apache.hadoop.metrics2.annotation.Metric(value=[GetGroups], about=, valueName=Time, type=DEFAULT, always=false, sampleName=Ops)

18/04/29 21:38:45 DEBUG lib.MutableMetricsFactory: field private org.apache.hadoop.metrics2.lib.MutableGaugeLong org.apache.hadoop.security.UserGroupInformation$UgiMetrics.renewalFailuresTotal with annotation @org.apache.hadoop.metrics2.annotation.Metric(value=[Renewal failures since startup], about=, valueName=Time, type=DEFAULT, always=false, sampleName=Ops)

18/04/29 21:38:45 DEBUG lib.MutableMetricsFactory: field private org.apache.hadoop.metrics2.lib.MutableGaugeInt org.apache.hadoop.security.UserGroupInformation$UgiMetrics.renewalFailures with annotation @org.apache.hadoop.metrics2.annotation.Metric(value=[Renewal failures since last successful login], about=, valueName=Time, type=DEFAULT, always=false, sampleName=Ops)

18/04/29 21:38:45 DEBUG impl.MetricsSystemImpl: UgiMetrics, User and group related metrics

18/04/29 21:38:45 DEBUG security.SecurityUtil: Setting hadoop.security.token.service.use_ip to true

18/04/29 21:38:45 DEBUG security.Groups:  Creating new Groups object

18/04/29 21:38:45 DEBUG util.NativeCodeLoader: Trying to load the custom-built native-hadoop library...

18/04/29 21:38:45 DEBUG util.NativeCodeLoader: Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: /usr/local/hadoop-2.9.0/lib/native/libhadoop.so.1.0.0: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /usr/local/hadoop-2.9.0/lib/native/libhadoop.so.1.0.0)

18/04/29 21:38:45 DEBUG util.NativeCodeLoader: java.library.path=/usr/local/hadoop-2.9.0/lib/native

18/04/29 21:38:45 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

18/04/29 21:38:45 DEBUG util.PerformanceAdvisory: Falling back to shell based

18/04/29 21:38:45 DEBUG security.JniBasedUnixGroupsMappingWithFallback: Group mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping

18/04/29 21:38:45 DEBUG security.Groups: Group mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; cacheTimeout=300000; warningDeltaMs=5000

18/04/29 21:38:45 DEBUG security.UserGroupInformation: hadoop login

18/04/29 21:38:45 DEBUG security.UserGroupInformation: hadoop login commit

18/04/29 21:38:45 DEBUG security.UserGroupInformation: using local user:UnixPrincipal: hadoop

18/04/29 21:38:45 DEBUG security.UserGroupInformation: Using user: "UnixPrincipal: hadoop" with name hadoop

18/04/29 21:38:45 DEBUG security.UserGroupInformation: User entry: "hadoop"

18/04/29 21:38:45 DEBUG security.UserGroupInformation: Assuming keytab is managed externally since logged in from subject.

18/04/29 21:38:45 DEBUG security.UserGroupInformation: UGI loginUser:hadoop (auth:SIMPLE)

18/04/29 21:38:45 DEBUG core.Tracer: sampler.classes = ; loaded no samplers

18/04/29 21:38:45 DEBUG core.Tracer: span.receiver.classes = ; loaded no span receivers

18/04/29 21:38:45 DEBUG fs.FileSystem: Loading filesystems

18/04/29 21:38:45 DEBUG fs.FileSystem: file:// = class org.apache.hadoop.fs.LocalFileSystem from /usr/local/hadoop-2.9.0/share/hadoop/common/hadoop-common-2.9.0.jar

18/04/29 21:38:46 DEBUG fs.FileSystem: viewfs:// = class org.apache.hadoop.fs.viewfs.ViewFileSystem from /usr/local/hadoop-2.9.0/share/hadoop/common/hadoop-common-2.9.0.jar

18/04/29 21:38:46 DEBUG fs.FileSystem: ftp:// = class org.apache.hadoop.fs.ftp.FTPFileSystem from /usr/local/hadoop-2.9.0/share/hadoop/common/hadoop-common-2.9.0.jar

18/04/29 21:38:46 DEBUG fs.FileSystem: har:// = class org.apache.hadoop.fs.HarFileSystem from /usr/local/hadoop-2.9.0/share/hadoop/common/hadoop-common-2.9.0.jar

18/04/29 21:38:46 DEBUG fs.FileSystem: http:// = class org.apache.hadoop.fs.http.HttpFileSystem from /usr/local/hadoop-2.9.0/share/hadoop/common/hadoop-common-2.9.0.jar

18/04/29 21:38:46 DEBUG fs.FileSystem: https:// = class org.apache.hadoop.fs.http.HttpsFileSystem from /usr/local/hadoop-2.9.0/share/hadoop/common/hadoop-common-2.9.0.jar

18/04/29 21:38:46 DEBUG fs.FileSystem: hdfs:// = class org.apache.hadoop.hdfs.DistributedFileSystem from /usr/local/hadoop-2.9.0/share/hadoop/hdfs/lib/hadoop-hdfs-client-2.9.0.jar

18/04/29 21:38:46 DEBUG fs.FileSystem: webhdfs:// = class org.apache.hadoop.hdfs.web.WebHdfsFileSystem from /usr/local/hadoop-2.9.0/share/hadoop/hdfs/lib/hadoop-hdfs-client-2.9.0.jar

18/04/29 21:38:46 DEBUG fs.FileSystem: swebhdfs:// = class org.apache.hadoop.hdfs.web.SWebHdfsFileSystem from /usr/local/hadoop-2.9.0/share/hadoop/hdfs/lib/hadoop-hdfs-client-2.9.0.jar

18/04/29 21:38:46 DEBUG fs.FileSystem: hftp:// = class org.apache.hadoop.hdfs.web.HftpFileSystem from /usr/local/hadoop-2.9.0/share/hadoop/hdfs/lib/hadoop-hdfs-client-2.9.0.jar

18/04/29 21:38:46 DEBUG fs.FileSystem: hsftp:// = class org.apache.hadoop.hdfs.web.HsftpFileSystem from /usr/local/hadoop-2.9.0/share/hadoop/hdfs/lib/hadoop-hdfs-client-2.9.0.jar

18/04/29 21:38:46 DEBUG fs.FileSystem: Looking for FS supporting hdfs

18/04/29 21:38:46 DEBUG fs.FileSystem: looking for configuration option fs.hdfs.impl

18/04/29 21:38:46 DEBUG fs.FileSystem: Looking in service filesystems for implementation class

18/04/29 21:38:46 DEBUG fs.FileSystem: FS for hdfs is class org.apache.hadoop.hdfs.DistributedFileSystem

18/04/29 21:38:46 DEBUG impl.DfsClientConf: dfs.client.use.legacy.blockreader.local = false

18/04/29 21:38:46 DEBUG impl.DfsClientConf: dfs.client.read.shortcircuit = false

18/04/29 21:38:46 DEBUG impl.DfsClientConf: dfs.client.domain.socket.data.traffic = false

18/04/29 21:38:46 DEBUG impl.DfsClientConf: dfs.domain.socket.path =

18/04/29 21:38:46 DEBUG hdfs.DFSClient: Sets dfs.client.block.write.replace-datanode-on-failure.min-replication to 0

18/04/29 21:38:46 DEBUG retry.RetryUtils: multipleLinearRandomRetry = null

18/04/29 21:38:46 DEBUG ipc.Server: rpcKind=RPC_PROTOCOL_BUFFER, rpcRequestWrapperClass=class org.apache.hadoop.ipc.ProtobufRpcEngine$RpcProtobufRequest, rpcInvoker=org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker@4977b5

18/04/29 21:38:46 DEBUG ipc.Client: getting client out of cache: org.apache.hadoop.ipc.Client@42583a89

18/04/29 21:38:47 DEBUG util.PerformanceAdvisory: Both short-circuit local reads and UNIX domain socket are disabled.

18/04/29 21:38:47 DEBUG sasl.DataTransferSaslUtil: DataTransferProtocol not using SaslPropertiesResolver, no QOP found in configuration for dfs.data.transfer.protection

18/04/29 21:38:47 DEBUG ipc.Client: The ping interval is 60000 ms.

18/04/29 21:38:47 DEBUG ipc.Client: Connecting to master/192.168.181.170:9000

18/04/29 21:38:47 DEBUG ipc.Client: IPC Client (136930299) connection to master/192.168.181.170:9000 from hadoop: starting, having connections 1

18/04/29 21:38:47 DEBUG ipc.Client: IPC Client (136930299) connection to master/192.168.181.170:9000 from hadoop sending #0 org.apache.hadoop.hdfs.protocol.ClientProtocol.getFileInfo

18/04/29 21:38:47 DEBUG ipc.Client: IPC Client (136930299) connection to master/192.168.181.170:9000 from hadoop got value #0

18/04/29 21:38:47 DEBUG ipc.ProtobufRpcEngine: Call: getFileInfo took 97ms

18/04/29 21:38:47 DEBUG ipc.Client: IPC Client (136930299) connection to master/192.168.181.170:9000 from hadoop sending #1 org.apache.hadoop.hdfs.protocol.ClientProtocol.getListing

18/04/29 21:38:47 DEBUG ipc.Client: IPC Client (136930299) connection to master/192.168.181.170:9000 from hadoop got value #1

18/04/29 21:38:47 DEBUG ipc.ProtobufRpcEngine: Call: getListing took 8ms

18/04/29 21:38:47 DEBUG ipc.Client: stopping client from cache: org.apache.hadoop.ipc.Client@42583a89

18/04/29 21:38:47 DEBUG ipc.Client: removing client from cache: org.apache.hadoop.ipc.Client@42583a89

18/04/29 21:38:47 DEBUG ipc.Client: stopping actual client because no more references remain: org.apache.hadoop.ipc.Client@42583a89

18/04/29 21:38:47 DEBUG ipc.Client: Stopping client

18/04/29 21:38:47 DEBUG ipc.Client: IPC Client (136930299) connection to master/192.168.181.170:9000 from hadoop: closed

18/04/29 21:38:47 DEBUG ipc.Client: IPC Client (136930299) connection to master/192.168.181.170:9000 from hadoop: stopped, remaining connections 0

18/04/29 21:38:47 DEBUG util.ShutdownHookManager: ShutdownHookManger complete shutdown.

发现报错说glibc-2.14版本找不到（被我框出的红色部分）。

 

$ strings /lib64/libc.so.6 | grep GLIBC     # 查看系统支持的版本，最高到2.12版本。

2、下载解压glibc-2.14版本；

 

1）下载地址：http://ftp.gnu.org/gnu/glibc/，找到glibc-2.14.tar.gz下载；

2）解压到任意路径下，我的解压路径/usr/local；

 

$  sudo tar -zxf ~/Downloads/hadoop-2.9.0.tar.gz -C /usr/local

 

2、在glibc源码目录下构建目录

 

$ cd glibc-2.14

$ mkdir build

$ cd build

3、运行configure配置,安装；

 

$ ../configure --prefix=/opt/glibc-2.14

$ sudo make -j4 # 这个时间可能稍微长些

$ sudo make install

4、配置

 

$ cp /etc/ld.so.c* /opt/glibc-2.14/etc/

cp: omitting directory `/etc/ld.so.conf.d'

$ sudo ln -sf /opt/glibc-2.14/lib/libc-2.14.so /lib64/libc.so.6

5、查看版本库的支持；

 

$ strings /lib64/libc.so.6 | grep GLIBC

 

参考链接：

https://www.cnblogs.com/likui360/p/6558749.html

https://blog.csdn.net/niying/article/details/42491557

https://www.cnblogs.com/erygreat/p/7223829.html

http://www.powerxing.com/install-hadoop-in-centos/