分布式系统datanode无法启动的报java.io.IOException: All specified directories are failed to load.

--java.io.IOException: All specified directories are failed to load.   概要


2018-04-18 01:52:54,822 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 50020: starting
2018-04-18 01:52:55,289 INFO org.apache.hadoop.hdfs.server.common.Storage: Lock on /opt/hadoop-2.7.2/current/data/in_use.lock acquired by nodename 1969@s201
2018-04-18 01:52:55,291 WARN org.apache.hadoop.hdfs.server.common.Storage: java.io.IOException: Incompatible clusterIDs in /opt/hadoop-2.7.2/current/data: namenode clusterID = CID-ca8a6a9a-7f14-4c71-9b17-299e0b500fd2; datanode clusterID = CID-45f6a315-e3c8-455e-b974-167e80bf7b65
2018-04-18 01:52:55,292 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for Block pool <registering> (Datanode Uuid unassigned) service to s200/192.168.30.200:9000. Exiting. 
java.io.IOException: All specified directories are failed to load.
at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:478)
at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:1358)
at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:1323)
at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:317)
at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:223)
at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:802)
at java.lang.Thread.run(Thread.java:748)
2018-04-18 01:52:55,296 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Ending block pool service for: Block pool <registering> (Datanode Uuid unassigned) service to s200/192.168.30.200:9000
2018-04-18 01:52:55,301 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Removed Block pool <registering> (Datanode Uuid unassigned)
2018-04-18 01:52:57,302 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Exiting Datanode
2018-04-18 01:52:57,303 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 0

2018-04-18 01:52:57,304 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG: 

==================解决办法===================
由于关机或者格式化文件系统的时候导致从日志上看是因为 datanode的clusterID 和 namenode的clusterID 不匹配造成的，解决办法找到namenode的current/dfs/name/VERSION （可能路径不一样）
然后找到clusterID，然后复制该ID到你的集群datanode上的VERSION文件里的clusterID，使得两个保持一致，重新启动集群，就好了


[root@s200 /opt/hadoop-2.7.2/current/dfs/name/current]# cat VERSION 
#Wed Apr 18 01:52:01 CST 2018
namespaceID=1328374604
clusterID=CID-ca8a6a9a-7f14-4c71-9b17-299e0b500fd2
cTime=0
storageType=NAME_NODE
blockpoolID=BP-1950255321-192.168.30.200-1523987521318
layoutVersion=-63
[root@s200 /opt/hadoop-2.7.2/current/dfs/name/current]#