bug宝典 hadoop篇 /hadoop/hdfs/data is in an inconsistent state: file VERSION has cTime missing.

最新推荐文章于 2023-03-08 11:55:00 发布

warrah

最新推荐文章于 2023-03-08 11:55:00 发布

阅读量1.5k

点赞数

分类专栏：岁月云——大数据杂烩岁月云——测试文章标签： bug hadoop

本文链接：https://blog.csdn.net/warrah/article/details/56009348

版权

岁月云——测试同时被 2 个专栏收录

78 篇文章 2 订阅

订阅专栏

岁月云——大数据杂烩

72 篇文章 3 订阅

订阅专栏

原本2个datanode，现在只看到了一个

从下图也可以看到只有一个live nodes，但是也没有dead node啊。

检查进程，缺少DataNode

[hadoop@dashuju172 ~]$ jps
48099 Main
53456 NodeManager
48363 Jps
48185 Main
54200 HRegionServer
53264 QuorumPeerMain

检查hadoop的日志

2017-02-20 09:52:49,741 WARN org.apache.hadoop.hdfs.server.common.Storage: org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /home/hadoop/platform/hadoop/hdfs/data is in an inconsistent state: file VERSION has cTime missing.
2017-02-20 09:52:49,742 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: 
Initialization failed for Block pool <registering> (Datanode Uuid unassigned) service to dashuju174/192.168.5.174:9000. Exiting. 
java.io.IOException: All specified directories are failed to load.
    at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:478)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:1338)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:1304)
    at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:314)
    at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:226)
    at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:867)
    at java.lang.Thread.run(Thread.java:745)
2017-02-20 09:52:49,745 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Ending block pool service for: 
Block pool <registering> (Datanode Uuid unassigned) service to dashuju174/192.168.5.174:9000
2017-02-20 09:52:49,846 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Removed Block pool <registering> (Datanode Uuid unassigned)
2017-02-20 09:52:51,847 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Exiting Datanode
2017-02-20 09:52:51,850 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 0
2017-02-20 09:52:51,852 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG:

检查datanode和namenode中
/home/hadoop/platform/hadoop/hdfs/name/current
/home/hadoop/platform/hadoop/hdfs/data/current中VERSION文件中clusterID
三台服务器是一致的啊，这就更奇怪了。

再看看FATAL前面的一句警告
WARN org.apache.hadoop.hdfs.server.common.Storage: org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /home/hadoop/platform/hadoop/hdfs/data is in an inconsistent state: file VERSION has cTime missing.
检查172的VERSION中确实没有cTime,修改后，执行start-dfs.sh,hadoop正常启动