HDFS 块损坏和恢复

HDFS 块损坏和恢复

# 先建个文件
echo 'The NameNode and DataNode are pieces of software designed to run on commodity machines. These machines typically run a GNU/Linux operating system (OS). HDFS is built using the Java language; any machine that supports Java can run the NameNode or the DataNode software. Usage of the highly portable Java language means that HDFS can be deployed on a wide range of machines. A typical deployment has a dedicated machine that runs only the NameNode software. Each of the other machines in the cluster runs one instance of the DataNode software. The architecture does not preclude running multiple DataNodes on the same machine but in a real deployment that is rarely the case.' > test.log

# 新建个目录
hdfs dfs -mkdir -p /recovery/test

# 上传文件
hdfs dfs -put ./test.log  /recovery/test/

#校验健康状态
hdfs fsck /
#############分割线#############
...................................Status: HEALTHY
 Total size:	215628 B
 Total dirs:	20
 Total files:	35
 Total symlinks:		0
 Total blocks (validated):	33 (avg. block size 6534 B)
 Minimally replicated blocks:	33 (100.0 %)
 Over-replicated blocks:	0 (0.0 %)
 Under-replicated blocks:	0 (0.0 %)
 Mis-replicated blocks:		0 (0.0 %)
 Default replication factor:	1
 Average block replication:	1.0
 Corrupt blocks:		0
 Missing replicas:		0 (0.0 %)
 Number of data-nodes:		1
 Number of racks:		1
FSCK ended at Sat Jul 06 17:46:49 EDT 2019 in 183 milliseconds
The filesystem under path '/' is HEALTHY
#############分割线 end#############

# 查看test.log block位置信息,BP-216010136-127.0.0.1-1561903454527:blk_1073741865_1041
hdfs fsck /recover/test/test.log -files -blocks -locations

# 删除一个block的副本
cd /tmp/hadoop-hadoop/dfs/data/current/BP-216010136-127.0.0.1-1561903454527/current/finalized/subdir0/subdir0
rm -rf blk_1073741862 blk_1073741862_1038.meta

# 再检查
hdfs fsck /
#############分割线#############
...................................Status: HEALTHY
...................................Status: HEALTHY
 Total size:	215628 B
 Total dirs:	20
 Total files:	35
 Total symlinks:		0
 Total blocks (validated):	33 (avg. block size 6534 B)
 Minimally replicated blocks:	33 (100.0 %)
 Over-replicated blocks:	0 (0.0 %)
 Under-replicated blocks:	0 (0.0 %)
 Mis-replicated blocks:		0 (0.0 %)
 Default replication factor:	1
 Average block replication:	1.0
 Corrupt blocks:		0
 Missing replicas:		0 (0.0 %)
 Number of data-nodes:		1
 Number of racks:		1
FSCK ended at Sun Jul 07 14:46:21 EDT 2019 in 138 milliseconds
The filesystem under path '/' is HEALTHY
#############分割线 end#############

# 额............好吧,啥也没变,听说伪分布式是没法恢复的,先就这样把,等到集群再说,先记录一下
# end

hdfs debug recoverLease -path /recovery/test/test.log -retries 10 recoverLease SUCCEEDED on /recovery/test/test.log

转:若泽大数据-生产HDFS Block损坏恢复最佳实践