1、Failed to start namenode.
重新格式化hadoop
查看logs日志
hdfs namenode –format
ormatting using clusterid: CID-dd578a63-1547-43fb-a4c4-4d058a9ab002
2021-12-06 18:43:13,572 ERROR namenode.NameNode: Failed to start namenode.
java.lang.NullPointerException
at java.io.File.<init>(File.java:277)
at org.apache.hadoop.hdfs.server.namenode.NNStorage.setStorageDirectories(NNStorage.java:320)
at org.apache.hadoop.hdfs.server.namenode.NNStorage.<init>(NNStorage.java:179)
at org.apache.hadoop.hdfs.server.namenode.FSImage.<init>(FSImage.java:167)
at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1206)
at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1673)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1783)
2021-12-06 18:43:13,587 INFO util.ExitUtil: Exiting with status 1: java.lang.NullPointerException
2021-12-06 18:43:13,619 INFO namenode.NameNode: SHUTDOWN_MSG:
检查hdfs-site.xml
这个错误是因为hdfs-site.xml文件配置错误导致的,仔细检查其中的 <name>dfs.namenode.shared.edits.dir</name>配置信息是否填写正确,一般都是因为这个配置信息写错导致的。
正确如下:
2、Failed to add storage directory
查看logs日志
021-12-06 18:58:13,948 WARN org.apache.hadoop.hdfs.server.common.Storage: Failed to add storage directory [DISK]file:/usr/local/hadoop/hadoopdata/datanode
java.io.IOException: Incompatible clusterIDs in /usr/local/hadoop/hadoopdata/datanode: namenode clusterID = CID-5cc3c497-58d4-4d06-a19d-71bdb88b13ce; datanode clusterID = CID-79134119-10c9-4381-a6a5-b4854ddb81b0
org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:829)
at java.lang.Thread.run(Thread.java:748)
2021-12-06 18:58:13,953 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for Block pool <registering> (Datanode Uuid a1081773-99a9-4a1a-a536-e5951fd35212) service to master/192.168.57.110:9000. Exiting.
java.io.IOException: All specified directories have failed to load.
删除data的current。
[root@hadoop-slave data]# pwd
/usr/local/hadoop-2.10.0/data/dfs/data
[root@hadoop-slave data]# ls
current
[root@hadoop-slave data]# rm -rf current
[root@hadoop-slave data]#
重启,检验Jps 6个结果
bash /usr/local/hadoop/sbin/stop-all.sh
bash /usr/local/hadoop/sbin/start-all.sh
[root@master logs]# jps
53089 NameNode
54099 Jps
53445 SecondaryNameNode
53224 DataNode
53832 NodeManager
53691 ResourceManager
3、测试报错
3.1、Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster
测试
./bin/hadoop jar ./share/hadoop/mapreduce/hadoop-mapreduce-examples-*.jar pi 3 3
结果:
处理方法,修改mapred-site.xml
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>mapreduce.application.classpath</name>
<value>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*</value>
</property>
<property>
<name>yarn.app.mapreduce.am.env</name>
<value>HADOOP_MAPRED_HOME=${HADOOP_HOME}</value>
</property>
<property>
<name>mapreduce.map.env</name>
<value>HADOOP_MAPRED_HOME=${HADOOP_HOME}</value>
</property>
<property>
<name>mapreduce.reduce.env</name>
<value>HADOOP_MAPRED_HOME=${HADOOP_HOME}</value>
</property>
</configuration>
3.2、节点启动之后没有NodeManager
bash /usr/local/hadoop/sbin/stop-all.sh
bash /usr/local/hadoop/sbin/start-all.sh
节点启动之后没有NodeManager
把slaves覆盖workers
Hadoop安装目录下的etc/hadoop/slaves文件,把slaves改成workers就可以生成日志了
Node1
Node2