jps不出现datanode_启动hadoop，jps没有datanode

启动./start-dfs.sh后jps发现没有datanode进程。

查看日志

2018-02-27 13:54:27,918 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting

2018-02-27 13:54:29,140 INFO org.apache.hadoop.hdfs.server.common.Storage: Lock on /home/hadoop/app/tmp/dfs/data/in_use.lock acquired by nodename 2873@hadoop000

2018-02-27 13:54:29,161 WARN org.apache.hadoop.hdfs.server.common.Storage: java.io.IOException: Incompatible clusterIDs in /home/hadoop/app/tmp/dfs/data: namenode clusterID = CID-d92efb85-6a10-4a65-abe8-451f89eb845c; datanode clusterID = CID-06da8613-d7f4-4bfa-ac2e-55104c7a265f

2018-02-27 13:54:29,162 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for Block pool (Datanode Uuid unassigned) service to hadoop000/192.168.92.128:8020. Exiting.

java.io.IOException: All specified directories are failed to load.

at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:478)

at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:1394)

at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:1355)

at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:317)

at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:228)

at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:829)

at java.lang.Thread.run(Thread.java:748)

2018-02-27 13:54:29,180 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Ending block pool service for: Block pool (Datanode Uuid unassigned) service to hadoop000/192.168.92.128:8020

2018-02-27 13:54:29,288 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Removed Block pool (Datanode Uuid unassigned)

2018-02-27 13:54:31,290 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Exiting Datanode

2018-02-27 13:54:31,292 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 0

2018-02-27 13:54:31,297 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG:

/************************************************************

SHUTDOWN_MSG: Shutting down DataNode at hadoop000/192.168.92.128

关键字

2018-02-27 13:54:29,161 WARN org.apache.hadoop.hdfs.server.common.Storage: java.io.IOException: Incompatible clusterIDs in /home/hadoop/app/tmp/dfs/data: namenode clusterID = CID-d92efb85-6a10-4a65-abe8-451f89eb845c; datanode clusterID = CID-06da8613-d7f4-4bfa-ac2e-55104c7a265f

从日志上看是因为 datanode的clusterID 和 namenode的clusterID 不匹配。

解决方法：

根据日志中的路径/home/hadoop/app/tmp/dfs/data，master可以看到name目录，salve可以看到data目录，

将name/current下的VERSION中的clusterID复制到data/current下的VERSION中，覆盖掉原来的clusterID，目的是让两个保持一致。

然后重启，就可以看到slave上的DataNode进程已经起来。

出现该问题的原因：

在第一次格式化dfs后，启动并使用了hadoop，后来又重新执行了格式化命令hdfs namenode -format，这时namenode的clusterID会重新生成，而datanode的clusterID保持不变。