配置hadoop集群常见报错汇总

配置hadoop集群常见报错汇总
1、使用hdfs namenode -format 格式化报错找不到JAVAHOME
该问题只需在对应的窗口导入JAVAHOME即可，注意，此处为对应环境安装的JDK路径，笔者为/usr/local/java
[hadoop@hadoop0 var]$ export JAVA_HOME=/usr/local/java
鉴于每次执行都要导入，建议直接在对应的/XXX/hadoop-xxx/etc/hadoop/hadoop-env.sh 添加如下语句，可以免去这个麻烦
export JAVA_HOME=/usr/local/java
export HADOOP_CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
2、使用hdfs namenode -format 格式化报错“Exception in thread "main" java.lang.UnsupportedClassVersionError: org/apache/hadoop/yarn/server/nodemanager/NodeManager : Unsupported major.minor version 52.0”
此问题为jdk版本过旧，hadoop3.x需要JDK1.8
解决措施：在JDK对应解压目录上删除旧的链接，重新建立JDK1.8的新链接
[root@hadoop2 ~]# cd /usr/local/
[root@hadoop2 local]# rm -f java
[root@hadoop2 local]# ln -sv jdk1.8.0_231 java
‘java’ -> ‘jdk1.7.0_80’
[root@hadoop2 local]# java -version
java version "1.8.0_231"
Java(TM) SE Runtime Environment (build 1.8.0_231-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.231-b11, mixed mode)

3、启动 start-all.sh 脚本发现如下一堆如下找不到命令的报错
/home/hadoop/hadoop-3.2.1/sbin/../libexec/hadoop-functions.sh: line 398: syntax error near unexpected token `<'
/home/hadoop/hadoop-3.2.1/sbin/../libexec/hadoop-functions.sh: line 398: `  done < <(for text in "${input[@]}"; do'
/home/hadoop/hadoop-3.2.1/sbin/../libexec/hadoop-config.sh: line 70: hadoop_deprecate_envvar: command not found
/home/hadoop/hadoop-3.2.1/sbin/../libexec/hadoop-config.sh: line 87: hadoop_bootstrap: command not found
/home/hadoop/hadoop-3.2.1/sbin/../libexec/hadoop-config.sh: line 104: hadoop_parse_args: command not found
/home/hadoop/hadoop-3.2.1/sbin/../libexec/hadoop-config.sh: line 105: shift: : numeric argument required
/home/hadoop/hadoop-3.2.1/sbin/../libexec/hadoop-config.sh: line 110: hadoop_find_confdir: command not found
/home/hadoop/hadoop-3.2.1/sbin/../libexec/hadoop-config.sh: line 111: hadoop_exec_hadoopenv: command not found
/home/hadoop/hadoop-3.2.1/sbin/../libexec/hadoop-config.sh: line 112: hadoop_import_shellprofiles: command not found
/home/hadoop/hadoop-3.2.1/sbin/../libexec/hadoop-config.sh: line 113: hadoop_exec_userfuncs: command not found
/home/hadoop/hadoop-3.2.1/sbin/../libexec/hadoop-config.sh: line 119: hadoop_exec_user_hadoopenv: command not found
/home/hadoop/hadoop-3.2.1/sbin/../libexec/hadoop-config.sh: line 120: hadoop_verify_confdir: command not found
/home/hadoop/hadoop-3.2.1/sbin/../libexec/hadoop-config.sh: line 122: hadoop_deprecate_envvar: command not found
/home/hadoop/hadoop-3.2.1/sbin/../libexec/hadoop-config.sh: line 123: hadoop_deprecate_envvar: command not found
/home/hadoop/hadoop-3.2.1/sbin/../libexec/hadoop-config.sh: line 124: hadoop_deprecate_envvar: command not found
/home/hadoop/hadoop-3.2.1/sbin/../libexec/hadoop-config.sh: line 129: hadoop_os_tricks: command not found
/home/hadoop/hadoop-3.2.1/sbin/../libexec/hadoop-config.sh: line 131: hadoop_java_setup: command not found
/home/hadoop/hadoop-3.2.1/sbin/../libexec/hadoop-config.sh: line 133: hadoop_basic_init: command not found
/home/hadoop/hadoop-3.2.1/sbin/../libexec/hadoop-config.sh: line 140: hadoop_shellprofiles_init: command not found
/home/hadoop/hadoop-3.2.1/sbin/../libexec/hadoop-config.sh: line 143: hadoop_add_javalibpath: command not found
/home/hadoop/hadoop-3.2.1/sbin/../libexec/hadoop-config.sh: line 144: hadoop_add_javalibpath: command not found
/home/hadoop/hadoop-3.2.1/sbin/../libexec/hadoop-config.sh: line 146: hadoop_shellprofiles_nativelib: command not found
/home/hadoop/hadoop-3.2.1/sbin/../libexec/hadoop-config.sh: line 152: hadoop_add_common_to_classpath: command not found
/home/hadoop/hadoop-3.2.1/sbin/../libexec/hadoop-config.sh: line 153: hadoop_shellprofiles_classpath: command not found
/home/hadoop/hadoop-3.2.1/sbin/../libexec/hadoop-config.sh: line 157: hadoop_exec_hadooprc: command not found
WARNING: Attempting to start all Apache Hadoop daemons as hadoop in 10 seconds.
WARNING: This is not a recommended production deployment configuration.
WARNING: Use CTRL-C to abort.
^C[hadoop@hadoop0 sbin]$
该问题可能是未配置程序运行的属主，也可能是执行脚本的shell错误
如下两步应可解决：
1）、 /XXX/hadoop-3.2.1/etc/hadoop/hadoop-env.sh 添加执行用户为hadoop，需已新建并配置免密
[hadoop@hadoop0 hadoop]$ tail hadoop-env.sh
# to only allow certain users to execute certain subcommands.
# It uses the for