hadoop集群启动后datanode没有启动

今天启动hadoop集群后，查看进程发现名称节点的NameNode进程启动了，但是数据节点的DataNode进程没有启动，如下：

master上
# jps 
27585 Jps
26874 ResourceManager
26622 NameNode

slave1上
# jps
25792 SecondaryNameNode
25988 Jps
25862 NodeManager

slave2上
# jps
25276 NodeManager
25581 Jps

原因为多次格式化namenode导致的namenode与datanode之间的不一致导致。

解决方法

（1）停止集群

# stop-all.sh 
This script is Deprecated. Instead use stop-dfs.sh and stop-yarn.sh
Stopping namenodes on [master]
master: stopping namenode
slave2: no datanode to stop
slave1: no datanode to stop
Stopping secondary namenodes [slave1]
slave1: stopping secondarynamenode
stopping yarn daemons
stopping resourcemanager
slave2: stopping nodemanager
slave1: stopping nodemanager
no proxyserver to stop

（2）删除datanode存放的路径的数据

可以在hdfs-site.xml中查看存放的路径，如下：

# vim /usr/local/hadoop/etc/hadoop/hdfs-site.xml

# rm -rf /usr/local/hadoop/data/*    #三个节点都操作

（3）重新格式化

# hadoop namenode -format

（4）启动集群

# start-all.sh 
This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
Starting namenodes on [master]
master: starting namenode, logging to /usr/local/hadoop-2.6.5/logs/hadoop-root-namenode-master.out
slave2: starting datanode, logging to /usr/local/hadoop-2.6.5/logs/hadoop-root-datanode-slave2.out
slave1: starting datanode, logging to /usr/local/hadoop-2.6.5/logs/hadoop-root-datanode-slave1.out
Starting secondary namenodes [slave1]
slave1: starting secondarynamenode, logging to /usr/local/hadoop-2.6.5/logs/hadoop-root-secondarynamenode-slave1.out
starting yarn daemons
starting resourcemanager, logging to /usr/local/hadoop-2.6.5/logs/yarn-root-resourcemanager-master.out
slave2: starting nodemanager, logging to /usr/local/hadoop-2.6.5/logs/yarn-root-nodemanager-slave2.out
slave1: starting nodemanager, logging to /usr/local/hadoop-2.6.5/logs/yarn-root-nodemanager-slave1.out

（4）查看进程

master 上
# jps -m 
28708 Jps -m
28380 ResourceManager
28126 NameNode

slave1上
# jps -m 
26848 Jps -m
26626 NodeManager
26557 SecondaryNameNode
26462 DataNode

slave2上
# jps -m 
25810 NodeManager
25996 Jps -m
25709 DataNode

（5）查看集群状态

# hadoop fsadmin -report
Error: Could not find or load main class fsadmin
[root@master hadoop]# hadoop dfsadmin -report
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.

Configured Capacity: 33335099392 (31.05 GB)
Present Capacity: 26244313088 (24.44 GB)
DFS Remaining: 26244263936 (24.44 GB)
DFS Used: 49152 (48 KB)
DFS Used%: 0.00%
Under replicated blocks: 0
Blocks with corrupt replicas: 0
Missing blocks: 0

-------------------------------------------------
Live datanodes (2):

Name: 192.168.22.129:50010 (slave1)
Hostname: slave1
Decommission Status : Normal
Configured Capacity: 16667549696 (15.52 GB)
DFS Used: 24576 (24 KB)
Non DFS Used: 3545563136 (3.30 GB)
DFS Remaining: 13121961984 (12.22 GB)
DFS Used%: 0.00%
DFS Remaining%: 78.73%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Tue Aug 27 00:02:44 CST 2019


Name: 192.168.22.130:50010 (slave2)
Hostname: slave2
Decommission Status : Normal
Configured Capacity: 16667549696 (15.52 GB)
DFS Used: 24576 (24 KB)
Non DFS Used: 3545223168 (3.30 GB)
DFS Remaining: 13122301952 (12.22 GB)
DFS Used%: 0.00%
DFS Remaining%: 78.73%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Tue Aug 27 00:02:44 CST 2019

查看进程，DataNode进程已正常启动，集群状态正常。