fsck
hdfsdatanodes
fsck
是一个HDFS
文件系统检查工具。
使用用法
hadoop fsck [[GENERIC_OPTIONS]] <path> [-move | -delete | -openforwrite] [-files [-blocks [-locations | -racks]]]
-
:检查的起始目录。
-
-move:移动受损文件到/lost+found
-
-delete:删除受损文件。
-
-files:打印出正被检查的文件。
-
-blocks: 打印出块信息报告。
-
-locations:打印出每个块的位置信息。
-
-racks:打印出data-node的网络拓扑结构。
使用案例
[zhangsan@node1 subdir0]$ hdfs fsck /user/zhangsan/hadoop-2.7.3.tar.gz -files -blocks -locations
Connecting to namenode via http://node1:50070/fsck?ugi=zhangsan&files=1&blocks=1&locations=1&path=%2Fuser%2Fzhangsan%2Fhadoop-2.7.3.tar.gz
FSCK started by zhangsan (auth:SIMPLE) from /192.168.179.101 for path /user/zhangsan/hadoop-2.7.3.tar.gz at Mon Apr 04 09:33:56 CST 2022
# 下面可以看到 文件、文件大小204.17MB、block数量、状态
/user/zhangsan/hadoop-2.7.3.tar.gz 214092195 bytes, 2 block(s): OK
# BP:Block Pool;
0. BP-792177474-192.168.179.101-1649033535606:blk_1073741825_1001
# len:block长度134217728 Byte, 即128MB;
len=134217728
# repl:replication = 3, 即副本数为3
repl=3
# 副本存放详细信息
[DatanodeInfoWithStorage[192.168.179.103:50010,DS-dfb7c7fc-8820-454b-a9d9-c84a257b1558,DISK],
DatanodeInfoWithStorage[192.168.179.102:50010,DS-7da4de31-9108-4c2b-95ae-f139922b7869,DISK],
DatanodeInfoWithStorage[192.168.179.101:50010,DS-e542e372-6db3-4c2f-90aa-0520df3e7066,DISK]]
# 第二个block
1. BP-792177474-192.168.179.101-1649033535606:blk_1073741826_1002
# 79874467(第二个数据块中的数据大小) = 214092195(文件总大小) - 134217728(第一个数据块中的数据大小)
len=79874467
repl=3
[DatanodeInfoWithStorage[192.168.179.102:50010,DS-7da4de31-9108-4c2b-95ae-f139922b7869,DISK],
DatanodeInfoWithStorage[192.168.179.103:50010,DS-dfb7c7fc-8820-454b-a9d9-c84a257b1558,DISK],
DatanodeInfoWithStorage[192.168.179.101:50010,DS-e542e372-6db3-4c2f-90aa-0520df3e7066,DISK]]
Status: HEALTHY
Total size: 214092195 B
Total dirs: 0
Total files: 1
Total symlinks: 0
Total blocks (validated): 2 (avg. block size 107046097 B)
Minimally replicated blocks: 2 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 3
Average block replication: 3.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 3
Number of racks: 1
FSCK ended at Mon Apr 04 09:33:56 CST 2022 in 1 milliseconds
The filesystem under path '/user/zhangsan/hadoop-2.7.3.tar.gz' is HEALTHY
https://hadoop.apache.org/docs/r1.0.4/cn/commands_manual.html#fsck