CDH5.8.4-Hadoop2.6.0安常见报错及解决方法

1.cdh2.6.0安装完成时报错

启动hadoop集群时报错，如下:

20/01/23 22:27:03 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

原因：导致该问题的改版本是因为${HADOOP_PREFIX}/lib/native目录没有lib库

2.检查命令

[root@host150 bigdata]# hadoop checknative -a
20/01/23 22:42:31 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Native library checking:
hadoop:  false 
zlib:    false 
snappy:  false 
lz4:     false 
bzip2:   false 
openssl: false 
20/01/23 22:42:31 INFO util.ExitUtil: Exiting with status 1

或者进入Hadoop下的/lib/native发现是空文件夹

3.解决办法

下载对应版本的native库，下载地址:

http://dl.bintray.com/sequenceiq/sequenceiq-bin/

下载完成之后解压到/lib/native目录:

[root@host152 native]# tar -xvf hadoop-native-64-2.6.0.tar

4.重启hdfs，再checknative查看

[root@host150 sbin]# hadoop checknative -a
20/01/23 23:16:35 WARN bzip2.Bzip2Factory: Failed to load/initialize native-bzip2 library system-native, will use pure-Java version
20/01/23 23:16:35 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library
Native library checking:
hadoop:  true /root/bigdata/hadoop/lib/native/libhadoop.so.1.0.0
zlib:    true /lib64/libz.so.1
snappy:  false 
lz4:     true revision:99
bzip2:   false 
openssl: false Cannot load libcrypto.so (libcrypto.so: cannot open shared object file: No such file or directory)!
20/01/23 23:16:35 INFO util.ExitUtil: Exiting with status 1

5.还是报错libcrypto.so错

openssl: false Cannot load libcrypto.so (libcrypto.so: cannot open shared object file: No such file or directory)!

解决:创建libcrypto.so软连接即可

[root@host150 sbin]# cd /usr/lib64/
[root@host150 lib64]# ln -s libcrypto.so.1.0.1e libcrypto.so

修改后再次checknative ，结果如下：

[root@host150 bigdata]# hadoop checknative -a
20/01/23 23:49:39 WARN bzip2.Bzip2Factory: Failed to load/initialize native-bzip2 library system-native, will use pure-Java version
20/01/23 23:49:39 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library
Native library checking:
hadoop:  true /root/bigdata/hadoop/lib/native/libhadoop.so.1.0.0
zlib:    true /lib64/libz.so.1
snappy:  false 
lz4:     true revision:99
bzip2:   false 
openssl: true /lib64/libcrypto.so
20/01/23 23:49:39 INFO util.ExitUtil: Exiting with status 1

说明

软链接：也称为符号链接,新建的文件以“路径”的形式来表示另一个文件，linux下的软链接类似于windows下的快捷方式。
　　举例:
       ln -s httpd.conf confighttp
　　其中 httpd.conf 就是源文件，confighttp是链接文件名,其作用是当编辑confighttp，实际上是链接编辑httpd.conf。
　　如果要删除软链接：
　　rm -rf confighttp
　　这样只会删除confighttp链接文件,而不会删除源文件httpd.conf。

硬链接：新建的文件是已经存在的文件的一个别名,当原文件删除时，新建的文件仍然可以使用,硬链接文件相当于一个文件存储在两个位置，可以有效防止误删。
　　ln httpd.conf confighttp

6.yarn启动之后页面看不到默认队列资源，使用资源显示NAN%

查看后台yarn日志报错如下：

2020-01-24 11:19:22,540 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 5 time(s); retry policy is RetryUpToMaximumCount
WithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2020-01-24 11:19:23,541 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 6 time(s); retry policy is RetryUpToMaximumCount
WithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)

解决办法:编辑yarn-site.xml，然后重启yarn集群即可。

[root@host152 hadoop]# vim yarn-site.xml

  <property>
    <name>yarn.resourcemanager.address</name>
    <value>host150:8032</value>
  </property>
  <property>
    <name>yarn.resourcemanager.scheduler.address</name>
    <value>host150:8030</value>
  </property>
  <property>
    <name>yarn.resourcemanager.resource-tracker.address</name>
    <value>host150:8031</value>
  </property>