查看 hdfs 上文件的文件状态、所属块信息等

执行 hdfs fsck 可以得到如下帮助信息：

Usage: hdfs fsck <path> [-list-corruptfileblocks | [-move | -delete | -openforwrite] [-files [-blocks [-locations | -racks]]]] [-includeSnapshots] [-storagepolicies] [-blockId <blk_Id>]
        <path>  start checking from this path
        -move   move corrupted files to /lost+found
        -delete delete corrupted files
        -files  显示被检查文件的状态信息
        -openforwrite   print out files opened for write
        -includeSnapshots    include snapshot data if the given path indicates a snapshottable directory or there are snapshottable directories under it
        -list-corruptfileblocks    print out list of missing blocks and files they belong to
        -blocks    显示被检查文件的块信息，需要和 -files 一起使用，即使用 -files -blocks 组合
        -locations    显示被检查文件块信息的位置，需要和 -files -blocks 一起使用，即 -files -blocks -locations 组合
        -racks    显示被检查文件的所属DataNode位置的网络拓扑，需要和 -files -blocks 一起使用，即使用 -files -blocks -racks 组合
        -storagepolicies    print out storage policy summary for the blocks
        -blockId    print out which file this blockId belongs to, locations (nodes, racks) of this block, and other diagnostics info (under replicated, corrupted or not, etc)

Please Note:
        1. By default fsck ignores files opened for write, use -openforwrite to report such files. They are usually  tagged CORRUPT or HEALTHY depending on their block allocation status
        2. Option -includeSnapshots should not be used for comparing stats, should be used only for HEALTH check, as this may contain duplicates if the same file present in both original fs tree and inside snapshots.

通过 hdfs fsck <path> [-files [-blocks [-locations | -racks]]]] 命令来获取文件的块信息。

-files

显示被检查文件的状态信息，示例：

[root@node1 ~]# hdfs fsck /hbase/data/default/testTable/208f33c214258061ed7f3afb796f42b5/.regioninfo -files
Connecting to namenode via http://192.169.14.102:50070/fsck?ugi=root&files=1&path=%2Fhbase%2Fdata%2Fdefault%2FtestTable%2F208f33c214258061ed7f3afb796f42b5%2F.regioninfo
FSCK started by root (auth:SIMPLE) from /192.169.14.102 for path /hbase/data/default/testTable/208f33c214258061ed7f3afb796f42b5/.regioninfo at Wed Jun 17 20:11:52 CST 2020
/hbase/data/default/testTable/208f33c214258061ed7f3afb796f42b5/.regioninfo 53 bytes, 1 block(s):  OK
Status: HEALTHY
 Total size:    53 B
 Total dirs:    0
 Total files:   1
 Total symlinks:                0
 Total blocks (validated):      1 (avg. block size 53 B)
 Minimally replicated blocks:   1 (100.0 %)
 Over-replicated blocks:        0 (0.0 %)
 Under-replicated blocks:       0 (0.0 %)
 Mis-replicated blocks:         0 (0.0 %)
 Default replication factor:    3
 Average block replication:     3.0
 Corrupt blocks:                0
 Missing replicas:              0 (0.0 %)
 Number of data-nodes:          3
 Number of racks:               1
FSCK ended at Wed Jun 17 20:11:52 CST 2020 in 1 milliseconds


The filesystem under path '/hbase/data/default/testTable/208f33c214258061ed7f3afb796f42b5/.regioninfo' is HEALTHY

关键信息：
/hbase/data/default/testTable/208f33c214258061ed7f3afb796f42b5/.regioninfo 53 bytes, 1 block(s):  OK

-blocks

显示被检查文件的块信息，包含 -files 的结果。需要和 -files 一起使用，即使用 -files -blocks 组合，示例

[root@node1 ~]# hdfs fsck /hbase/data/default/testTable/208f33c214258061ed7f3afb796f42b5/.regioninfo -files -blocks
Connecting to namenode via http://192.169.14.102:50070/fsck?ugi=root&files=1&blocks=1&path=%2Fhbase%2Fdata%2Fdefault%2FtestTable%2F208f33c214258061ed7f3afb796f42b5%2F.regioninfo
FSCK started by root (auth:SIMPLE) from /192.169.14.102 for path /hbase/data/default/testTable/208f33c214258061ed7f3afb796f42b5/.regioninfo at Wed Jun 17 20:15:50 CST 2020
/hbase/data/default/testTable/208f33c214258061ed7f3afb796f42b5/.regioninfo 53 bytes, 1 block(s):  OK
0. BP-1491845419-192.169.14.102-1592298954391:blk_1073742018_1194 len=53 repl=3

Status: HEALTHY
 Total size:    53 B
 Total dirs:    0
 Total files:   1
 Total symlinks:                0
 Total blocks (validated):      1 (avg. block size 53 B)
 Minimally replicated blocks:   1 (100.0 %)
 Over-replicated blocks:        0 (0.0 %)
 Under-replicated blocks:       0 (0.0 %)
 Mis-replicated blocks:         0 (0.0 %)
 Default replication factor:    3
 Average block replication:     3.0
 Corrupt blocks:                0
 Missing replicas:              0 (0.0 %)
 Number of data-nodes:          3
 Number of racks:               1
FSCK ended at Wed Jun 17 20:15:50 CST 2020 in 1 milliseconds


The filesystem under path '/hbase/data/default/testTable/208f33c214258061ed7f3afb796f42b5/.regioninfo' is HEALTHY

关键信息：
/hbase/data/default/testTable/208f33c214258061ed7f3afb796f42b5/.regioninfo 53 bytes, 1 block(s):  OK
0. BP-1491845419-192.169.14.102-1592298954391:blk_1073742018_1194 len=53 repl=3

-locations

显示被检查文件块信息的位置。需要和 -files -blocks 一起使用，即使用 -files -blocks -locations 组合。

[root@node1 ~]# hdfs fsck /hbase/data/default/testTable/208f33c214258061ed7f3afb796f42b5/.regioninfo -files -blocks -locations
Connecting to namenode via http://192.169.14.102:50070/fsck?ugi=root&files=1&blocks=1&locations=1&path=%2Fhbase%2Fdata%2Fdefault%2FtestTable%2F208f33c214258061ed7f3afb796f42b5%2F.regioninfo
FSCK started by root (auth:SIMPLE) from /192.169.14.102 for path /hbase/data/default/testTable/208f33c214258061ed7f3afb796f42b5/.regioninfo at Wed Jun 17 20:18:06 CST 2020
/hbase/data/default/testTable/208f33c214258061ed7f3afb796f42b5/.regioninfo 53 bytes, 1 block(s):  OK
0. BP-1491845419-192.169.14.102-1592298954391:blk_1073742018_1194 len=53 repl=3 [DatanodeInfoWithStorage[192.169.14.119:50010,DS-0b9060b7-ab34-4c65-80f6-e308857237ae,DISK], DatanodeInfoWithStorage[192.169.14.103:50010,DS-afc0f984-1fa5-4372-965f-9cc0a1bcf751,DISK], DatanodeInfoWithStorage[192.169.14.102:50010,DS-6f7deeae-4daa-4d4b-9e8f-eedd3f3455af,DISK]]

Status: HEALTHY
 Total size:    53 B
 Total dirs:    0
 Total files:   1
 Total symlinks:                0
 Total blocks (validated):      1 (avg. block size 53 B)
 Minimally replicated blocks:   1 (100.0 %)
 Over-replicated blocks:        0 (0.0 %)
 Under-replicated blocks:       0 (0.0 %)
 Mis-replicated blocks:         0 (0.0 %)
 Default replication factor:    3
 Average block replication:     3.0
 Corrupt blocks:                0
 Missing replicas:              0 (0.0 %)
 Number of data-nodes:          3
 Number of racks:               1
FSCK ended at Wed Jun 17 20:18:06 CST 2020 in 1 milliseconds


The filesystem under path '/hbase/data/default/testTable/208f33c214258061ed7f3afb796f42b5/.regioninfo' is HEALTHY

关键信息：
/hbase/data/default/testTable/208f33c214258061ed7f3afb796f42b5/.regioninfo 53 bytes, 1 block(s):  OK
0. BP-1491845419-192.169.14.102-1592298954391:blk_1073742018_1194 len=53 repl=3 [DatanodeInfoWithStorage[192.169.14.119:50010,DS-0b9060b7-ab34-4c65-80f6-e308857237ae,DISK], DatanodeInfoWithStorage[192.169.14.103:50010,DS-afc0f984-1fa5-4372-965f-9cc0a1bcf751,DISK], DatanodeInfoWithStorage[192.169.14.102:50010,DS-6f7deeae-4daa-4d4b-9e8f-eedd3f3455af,DISK]] 

-racks

显示被检查文件的所属DataNode位置的网络拓扑。需要和 -files -blocks 一起使用，即使用 -files -blocks -racks 组合。

[root@node1 ~]# hdfs fsck /hbase/data/default/testTable/208f33c214258061ed7f3afb796f42b5/.regioninfo -files -blocks -racks
Connecting to namenode via http://192.169.14.102:50070/fsck?ugi=root&files=1&blocks=1&racks=1&path=%2Fhbase%2Fdata%2Fdefault%2FtestTable%2F208f33c214258061ed7f3afb796f42b5%2F.regioninfo
FSCK started by root (auth:SIMPLE) from /192.169.14.102 for path /hbase/data/default/testTable/208f33c214258061ed7f3afb796f42b5/.regioninfo at Wed Jun 17 20:19:21 CST 2020
/hbase/data/default/testTable/208f33c214258061ed7f3afb796f42b5/.regioninfo 53 bytes, 1 block(s):  OK
0. BP-1491845419-192.169.14.102-1592298954391:blk_1073742018_1194 len=53 repl=3 [/default-rack/192.169.14.119:50010, /default-rack/192.169.14.103:50010, /default-rack/192.169.14.102:50010]

Status: HEALTHY
 Total size:    53 B
 Total dirs:    0
 Total files:   1
 Total symlinks:                0
 Total blocks (validated):      1 (avg. block size 53 B)
 Minimally replicated blocks:   1 (100.0 %)
 Over-replicated blocks:        0 (0.0 %)
 Under-replicated blocks:       0 (0.0 %)
 Mis-replicated blocks:         0 (0.0 %)
 Default replication factor:    3
 Average block replication:     3.0
 Corrupt blocks:                0
 Missing replicas:              0 (0.0 %)
 Number of data-nodes:          3
 Number of racks:               1
FSCK ended at Wed Jun 17 20:19:21 CST 2020 in 1 milliseconds


The filesystem under path '/hbase/data/default/testTable/208f33c214258061ed7f3afb796f42b5/.regioninfo' is HEALTHY

关键信息：
/hbase/data/default/testTable/208f33c214258061ed7f3afb796f42b5/.regioninfo 53 bytes, 1 block(s):  OK
0. BP-1491845419-192.169.14.102-1592298954391:blk_1073742018_1194 len=53 repl=3 [/default-rack/192.169.14.119:50010, /default-rack/192.169.14.103:50010, /default-rack/192.169.14.102:50010] 

结论

如果想要查看文件的文件状态和所属的块信息，则使用如下命令：
hdfs fsck <path> -files -blocks -locations

如果想要查看文件的文件状态和所属的DataNode网络拓扑，则使用如下命令：
hdfs fsck <path> -files -blocks -racks