Hadoop集群有可能出现的错误

在新搭建的hadoop集群中出现了许多问题，今天为大家带来我在运行新搭建的集群时候，出现的一些问题，希望可以帮助到你们。

第一个

在使用启动集群的命令start-dfs.sh时，出现下面的警告：

WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

问题在哪里？有人说这是hadoop的预编译包是32bit的，运行在64bit上就会有问题。但是这个答案大多数时候都是错的。
如何验证64bit还是32bit?

进入hadoop安装的目录

/home/hadoop/apps/hadoop-2.6.4/lib/native

用ldd命令查看依赖库

ldd libhadoop.so.1.0.0

会输出如下信息：

./libhadoop.so.1.0.0: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by ./libhadoop.so.1.0.0)
        linux-vdso.so.1 =>  (0x00007fff369ff000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f3caa7ea000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f3caa455000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f3caac1b000)

可以看到依赖的都是/lib64/的动态库，所以不是64位/32位问题。但是看到报错，GLIBC_2.14找不到，现在检查系统的glibc库, ldd –version即可检查。
输入命令：

ldd --version

会输出如下信息：

[root@admin1 native]# ldd --version
ldd (GNU libc) 2.12
Copyright (C) 2010 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
由 Roland McGrath 和 Ulrich Drepper 编写。

原来系统预装的glibc库是2.12版本，而hadoop期望是2.14版本，所以打印警告信息。

现在有两个办法，重新编译glibc.2.14版本，安装后专门给hadoop使用，这个有点危险。

第二个办法直接在log4j日志中去除告警信息。在//usr/local/hadoop-2.5.2/etc/hadoop/log4j.properties文件中添加

log4j.logger.org.apache.hadoop.util.NativeCodeLoader=ERROR

第二个

同样是使用start-dfs.sh命令时，出现下面的错误：

-c: Unknown cipher type 'cd'
admin1: stopping namenode
try: ssh: Could not resolve hostname try: Name or service not known
disabled: ssh: Could not resolve hostname disabled: Name or service not known
VM: ssh: Could not resolve hostname VM: Name or service not known
will: ssh: Could not resolve hostname will: Name or service not known
recommended: ssh: Could not resolve hostname recommended: Name or service not known
It's: ssh: Could not resolve hostname It's: Name or service not known
HotSpot(TM): ssh: Could not resolve hostname HotSpot(TM): Name or service not known
have: ssh: Could not resolve hostname have: Name or service not known
The: ssh: Could not resolve hostname The: Name or service not known
VM: ssh: Could not resolve hostname VM: Name or service not known
which: ssh: Could not resolve hostname which: Name or service not known
have: ssh: Could not resolve hostname have: Name or service not known
might: ssh: Could not resolve hostname might: Name or service not known
Client: ssh: Could not resolve hostname Client: Name or service not known
the: ssh: Could not resolve hostname the: Name or service not known
highly: ssh: Could not resolve hostname highly: Name or service not known
that: ssh: Could not resolve hostname that: Name or service not known
'execstack: ssh: Could not resolve hostname 'execstack: Name or service not known
stack: ssh: Could not resolve hostname stack: Name or service not known
guard: ssh: Could not resolve hostname guard: Name or service not known
library: ssh: Could not resolve hostname library: Name or service not known
guard.: ssh: Could not resolve hostname guard.: Name or service not known
You: ssh: Could not resolve hostname You: Name or service not known
warning:: ssh: Could not resolve hostname warning:: Name or service not known
loaded: ssh: Could not resolve hostname loaded: Name or service not known
fix: ssh: Could not resolve hostname fix: Name or service not known
it: ssh: Could not resolve hostname it: Name or service not known
to: ssh: Could not resolve hostname to: Name or service not known
stack: ssh: Could not resolve hostname stack: Name or service not known
Java: ssh: Could not resolve hostname Java: Name or service not known
now.: ssh: Could not resolve hostname now.: Name or service not known
library: ssh: Could not resolve hostname library: Name or service not known
the: ssh: Could not resolve hostname the: Name or service not known
you: ssh: Could not resolve hostname you: Name or service not known
fix: ssh: Could not resolve hostname fix: Name or service not known
'-z: ssh: Could not resolve hostname '-z: Name or service not known
or: ssh: Could not resolve hostname or: Name or service not known
with: ssh: Could not resolve hostname with: Name or service not known
<libfile>',: ssh: Could not resolve hostname <libfile>',: Name or service not known
link: ssh: Could not resolve hostname link: Name or service not known
noexecstack'.: ssh: Could not resolve hostname noexecstack'.: Name or service not known
with: ssh: Could not resolve hostname with: Name or service not known

解决方法：
在/etc/profile文件中添加：

export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HADOOP_OPTS=-Djava.library.path=$HADOOP_HOME/lib

值得注意：是export HADOOP_OPTS=-Djava.library.path=$HADOOP_HOME/lib，而不是export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib"，还需要注意的是在复制或者编写的时候，一定要将书写好的代码检查清楚看是否缺少什么字母，如果缺少，那么你将有可能会遇到下面这个问题。

第三个

在编写/etc/profiel里面的文件时候，一定要注意检查看是否有字母错误，否则将会出现下面的错误：

[root@admin1 ~]# source /etc/profile
bash: ort: command not found
[root@admin1 ~]#