hdfs fsck

fsck(file system check) ,用于检查HDFS上文件和目录的健康状态、获取文件的block块信息和位置信息等。

#hdfs dfs -ls /user/gewei/
#hdfs dfs -rm /user/gewei/houseApplyTest.csv
hadoop fs -put houseApplyTest.csv  hdfs:/user/gewei

[dev@cdh1 python]$ hdfs dfs -ls /user/gewei/houseApplyTest.csv
-rw-r--r--   3 dev supergroup  600000000 2019-02-28 10:49 /user/gewei/houseApplyTest.csv

[dev@cdh1 python]$ hadoop fsck /user/gewei/houseApplyTest.csv -files -blocks -locations
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.

Connecting to namenode via http://cdh1:50070/fsck?ugi=dev&files=1&blocks=1&locations=1&path=%2Fuser%2Fgewei%2FhouseApplyTest.csv
FSCK started by dev (auth:SIMPLE) from /10.3.1.8 for path /user/gewei/houseApplyTest.csv at Thu Feb 28 16:34:00 CST 2019
/user/gewei/houseApplyTest.csv 600000000 bytes, 5 block(s):  OK
0. BP-1123673671-10.3.1.8-1513763234376:blk_1075741292_2000552 len=134217728 Live_repl=3 [DatanodeInfoWithStorage[10.3.1.13:50010,DS-cf291cd2-a872-490d-8c70-52f6ce58399a,DISK], DatanodeInfoWithStorage[10.3.1.9:50010,DS-36a9eb13-f4a3-4ba8-bf84-a53e081ebb89,DISK], DatanodeInfoWithStorage[10.3.1.14:50010,DS-c2ae84aa-516f-4e1f-b255-76a00a7cc0f2,DISK]]
1. BP-1123673671-10.3.1.8-1513763234376:blk_1075741293_2000553 len=134217728 Live_repl=3 [DatanodeInfoWithStorage[10.3.1.13:50010,DS-cf291cd2-a872-490d-8c70-52f6ce58399a,DISK], DatanodeInfoWithStorage[10.3.1.9:50010,DS-36a9eb13-f4a3-4ba8-bf84-a53e081ebb89,DISK], DatanodeInfoWithStorage[10.3.1.14:50010,DS-c2ae84aa-516f-4e1f-b255-76a00a7cc0f2,DISK]]
2. BP-1123673671-10.3.1.8-1513763234376:blk_1075741294_2000554 len=134217728 Live_repl=3 [DatanodeInfoWithStorage[10.3.1.14:50010,DS-c2ae84aa-516f-4e1f-b255-76a00a7cc0f2,DISK], DatanodeInfoWithStorage[10.3.1.9:50010,DS-36a9eb13-f4a3-4ba8-bf84-a53e081ebb89,DISK], DatanodeInfoWithStorage[10.3.1.13:50010,DS-cf291cd2-a872-490d-8c70-52f6ce58399a,DISK]]
3. BP-1123673671-10.3.1.8-1513763234376:blk_1075741295_2000555 len=134217728 Live_repl=3 [DatanodeInfoWithStorage[10.3.1.9:50010,DS-36a9eb13-f4a3-4ba8-bf84-a53e081ebb89,DISK], DatanodeInfoWithStorage[10.3.1.14:50010,DS-c2ae84aa-516f-4e1f-b255-76a00a7cc0f2,DISK], DatanodeInfoWithStorage[10.3.1.13:50010,DS-cf291cd2-a872-490d-8c70-52f6ce58399a,DISK]]
4. BP-1123673671-10.3.1.8-1513763234376:blk_1075741296_2000556 len=63129088 Live_repl=3 [DatanodeInfoWithStorage[10.3.1.13:50010,DS-cf291cd2-a872-490d-8c70-52f6ce58399a,DISK], DatanodeInfoWithStorage[10.3.1.9:50010,DS-36a9eb13-f4a3-4ba8-bf84-a53e081ebb89,DISK], DatanodeInfoWithStorage[10.3.1.14:50010,DS-c2ae84aa-516f-4e1f-b255-76a00a7cc0f2,DISK]]

Status: HEALTHY
 Total size:	600000000 B
 Total dirs:	0
 Total files:	1
 Total symlinks:		0
 Total blocks (validated):	5 (avg. block size 120000000 B)
 Minimally replicated blocks:	5 (100.0 %)
 Over-replicated blocks:	0 (0.0 %)
 Under-replicated blocks:	0 (0.0 %)
 Mis-replicated blocks:		0 (0.0 %)
 Default replication factor:	3
 Average block replication:	3.0
 Corrupt blocks:		0
 Missing replicas:		0 (0.0 %)
 Number of data-nodes:		3
 Number of racks:		1
FSCK ended at Thu Feb 28 16:34:00 CST 2019 in 1 milliseconds


The filesystem under path '/user/gewei/houseApplyTest.csv' is HEALTHY

gaiwei@moerlong.com