Hadoop: hdfs 块异常检查

在hdfs上创建了一个文件

[hadoop@hdp02 ~]$ echo "hadoop file" > hadoop01.txt
[hadoop@hdp02 ~]$ hadoop fs -mkdir /hadoop
[hadoop@hdp02 ~]$ hadoop fs -put hadoop01.txt /hadoop/
[hadoop@hdp02 ~]$ hadoop fs -ls -h /hadoop
Found 1 items
-rw-r--r--   3 hadoop supergroup         19 2020-04-05 21:02 /hadoop/hadoop01.txt

 使用fsck命令进行hdfs健康检查，发现了文件异常，配置副本为3，只有2副本

#hdfs文件健康检查
[hadoop@hdp02 ~]$ hdfs fsck /
Connecting to namenode via http://hdp01:50070/fsck?ugi=hadoop&path=%2F
FSCK started by hadoop (auth:SIMPLE) from /192.168.60.101 for path / at Sun Apr 05 22:53:38 CST 2020
.
/hadoop/hadoop01.txt:  Under replicated BP-417206835-192.168.60.101-1586085282109:blk_1073741825_1001. Target Replicas is 3 but found 2 live replica(s), 0 decommissioned replica(s), 0 decommissioning replica(s).
Status: HEALTHY
 Total size:    12 B
 Total dirs:    2
 Total files:   1
 Total symlinks:                0
 Total blocks (validated):      1 (avg. block size 12 B)
 Minimally replicated blocks:   1 (100.0 %)
 Over-replicated blocks:        0 (0.0 %)
 Under-replicated blocks:       1 (100.0 %)
 Mis-replicated blocks:         0 (0.0 %)
 Default replication factor:    3
 Average block replication:     2.0
 Corrupt blocks:                0
 Missing replicas:              1 (33.333332 %)
 Number of data-nodes:          2
 Number of racks:               1
FSCK ended at Sun Apr 05 22:53:38 CST 2020 in 1 milliseconds


The filesystem under path '/' is indeed HEALTHY

异常信息显示，一个hdfs 文件块丢失

执行hdfs文件手动恢复命令

hdfs debug recoverLease -path /hadoop/hadoop01.txt -retries 10

然后再查看健康状况，执行hdfs fsck，发现信息还是和上面一样。问题出在哪？

 

后来想想，我的集群只配置了一台namenode和2台datanode，文件副本只会存储在datanode节点上。所以就算我的hdfs-site.xml文件配置了副本数为3，实际也最多只能存在2个副本！

 

没办法，需要增加一个datanode节点。编辑 $HADOOP_HOME/etc/hadoop/slaves文件，增加hdp01为新的datanode节点，然后将该文件同步到其他节点。

 

增加了datanode节点，集群性质发生了变化，需要重新初始化namenode，格式化需要进行一系列步骤，否则格式化无效！

如何格式初始化namenode，参见我的 hdfs datanode进程没法启动

 

完成格式化后，创建新的文件

再次执行hdfs fsck / 查看hdfs 文件系统状况

[hadoop@hdp02 hadoop]$ hadoop fs -put README.txt /hadoop/
[hadoop@hdp02 hadoop]$ hdfs fsck /
Connecting to namenode via http://hdp01:50070/fsck?ugi=hadoop&path=%2F
FSCK started by hadoop (auth:SIMPLE) from /192.168.60.102 for path / at Mon Apr 06 00:01:24 CST 2020
.Status: HEALTHY
 Total size:    1366 B
 Total dirs:    2
 Total files:   1
 Total symlinks:                0
 Total blocks (validated):      1 (avg. block size 1366 B)
 Minimally replicated blocks:   1 (100.0 %)
 Over-replicated blocks:        0 (0.0 %)
 Under-replicated blocks:       0 (0.0 %)
 Mis-replicated blocks:         0 (0.0 %)
 Default replication factor:    3
 Average block replication:     3.0
 Corrupt blocks:                0
 Missing replicas:              0 (0.0 %)
 Number of data-nodes:          3
 Number of racks:               1
FSCK ended at Mon Apr 06 00:01:24 CST 2020 in 5 milliseconds


The filesystem under path '/' is HEALTHY