Hdfs 故障恢复

最新推荐文章于 2021-12-24 20:44:24 发布
最新推荐文章于 2021-12-24 20:44:24 发布
阅读量1k 收藏
点赞数
分类专栏： Hadoop
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/xiaoxiongaa0/article/details/89053695
版权

 下面以伪分布式为例来进行故障恢复。ok，let's go

[hadoop@hadoop001 hadoop-2.6.0-cdh5.7.0]$ hdfs fsck /
19/04/06 11:40:44 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Connecting to namenode via http://hadoop001:50070
FSCK started by hadoop (auth:SIMPLE) from /172.18.32.186 for path / at Sat Apr 06 11:40:45 CST 2019
.
/g6/hadoop/access/clear/day=20180717/part-r-00000: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742082

/g6/hadoop/access/clear/day=20180717/part-r-00000: MISSING 1 blocks of total size 9116696 B...
/g6/hadoop/accesslog/20180717/test.log: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742070

/g6/hadoop/accesslog/20180717/test.log: MISSING 1 blocks of total size 43282330 B..
/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0002/job.jar: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742063

/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0002/job.jar: MISSING 1 blocks of total size 6732 B..
/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0002/job.split: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742064

/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0002/job.split: MISSING 1 blocks of total size 4099 B..
/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0002/job.splitmetainfo: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742065

/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0002/job.splitmetainfo: MISSING 1 blocks of total size 633 B..
/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0002/job.xml: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742066

/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0002/job.xml: MISSING 1 blocks of total size 91915 B..
/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0002/job_1553580778554_0002.summary: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742068

/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0002/job_1553580778554_0002.summary: MISSING 1 blocks of total size 297 B..
/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0002/job_1553580778554_0002_1.jhist: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742069

/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0002/job_1553580778554_0002_1.jhist: MISSING 1 blocks of total size 42661 B..
/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0002/job_1553580778554_0002_1_conf.xml: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742067

/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0002/job_1553580778554_0002_1_conf.xml: MISSING 1 blocks of total size 109349 B..
/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0003/job.jar: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742071

/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0003/job.jar: MISSING 1 blocks of total size 6732 B..
/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0003/job.split: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742072

/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0003/job.split: MISSING 1 blocks of total size 131 B..
/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0003/job.splitmetainfo: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742073

/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0003/job.splitmetainfo: MISSING 1 blocks of total size 27 B..
/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0003/job.xml: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742074

/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0003/job.xml: MISSING 1 blocks of total size 91914 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1550983677568_0001-1550984046181-hadoop-QuasiMonteCarlo-1550984067738-5-1-SUCCEEDED-root.hadoop-1550984051208.jhist: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741840

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1550983677568_0001-1550984046181-hadoop-QuasiMonteCarlo-1550984067738-5-1-SUCCEEDED-root.hadoop-1550984051208.jhist: MISSING 1 blocks of total size 59168 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1550983677568_0001.summary: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741839

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1550983677568_0001.summary: MISSING 1 blocks of total size 355 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1550983677568_0001_conf.xml: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741841

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1550983677568_0001_conf.xml: MISSING 1 blocks of total size 110148 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1550983677568_0002-1550984832078-hadoop-word+count-1550984847506-2-1-SUCCEEDED-root.hadoop-1550984836666.jhist: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741853

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1550983677568_0002-1550984832078-hadoop-word+count-1550984847506-2-1-SUCCEEDED-root.hadoop-1550984836666.jhist: MISSING 1 blocks of total size 40208 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1550983677568_0002.summary: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741852

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1550983677568_0002.summary: MISSING 1 blocks of total size 349 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1550983677568_0002_conf.xml: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741854

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1550983677568_0002_conf.xml: MISSING 1 blocks of total size 109840 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0001-1551595710002-hadoop-select+word%2C+count%281%29+c%0Afrom%0A%28select+...desc%28Stage-1551595727139-1-1-SUCCEEDED-root.hadoop-1551595716018.jhist: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741867

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0001-1551595710002-hadoop-select+word%2C+count%281%29+c%0Afrom%0A%28select+...desc%28Stage-1551595727139-1-1-SUCCEEDED-root.hadoop-1551595716018.jhist: MISSING 1 blocks of total size 35410 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0001.summary: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741866

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0001.summary: MISSING 1 blocks of total size 395 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0001_conf.xml: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741868

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0001_conf.xml: MISSING 1 blocks of total size 222878 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0002-1551595730185-hadoop-select+word%2C+count%281%29+c%0Afrom%0A%28select+...desc%28Stage-1551595744690-1-1-SUCCEEDED-root.hadoop-1551595734416.jhist: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741880

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0002-1551595730185-hadoop-select+word%2C+count%281%29+c%0Afrom%0A%28select+...desc%28Stage-1551595744690-1-1-SUCCEEDED-root.hadoop-1551595734416.jhist: MISSING 1 blocks of total size 35475 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0002.summary: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741879

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0002.summary: MISSING 1 blocks of total size 395 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0002_conf.xml: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741881

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0002_conf.xml: MISSING 1 blocks of total size 223248 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0003-1551595813288-hadoop-select+word%2C+count%281%29+c+from+%28select+...desc%28Stage-1551595828436-1-1-SUCCEEDED-root.hadoop-1551595817375.jhist: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741893

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0003-1551595813288-hadoop-select+word%2C+count%281%29+c+from+%28select+...desc%28Stage-1551595828436-1-1-SUCCEEDED-root.hadoop-1551595817375.jhist: MISSING 1 blocks of total size 35403 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0003.summary: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741892

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0003.summary: MISSING 1 blocks of total size 393 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0003_conf.xml: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741894

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0003_conf.xml: MISSING 1 blocks of total size 222876 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0004-1551595830949-hadoop-select+word%2C+count%281%29+c+from+%28select+...desc%28Stage-1551595845881-1-1-SUCCEEDED-root.hadoop-1551595835253.jhist: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741906

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0004-1551595830949-hadoop-select+word%2C+count%281%29+c+from+%28select+...desc%28Stage-1551595845881-1-1-SUCCEEDED-root.hadoop-1551595835253.jhist: MISSING 1 blocks of total size 35479 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0004.summary: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741905

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0004.summary: MISSING 1 blocks of total size 393 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0004_conf.xml: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741907

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0004_conf.xml: MISSING 1 blocks of total size 223246 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0005-1551595975632-hadoop-select+word%2C+count%281%29+c+from+%28select+...desc%28Stage-1551595991154-1-1-SUCCEEDED-root.hadoop-1551595979967.jhist: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741919

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0005-1551595975632-hadoop-select+word%2C+count%281%29+c+from+%28select+...desc%28Stage-1551595991154-1-1-SUCCEEDED-root.hadoop-1551595979967.jhist: MISSING 1 blocks of total size 35403 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0005.summary: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741918

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0005.summary: MISSING 1 blocks of total size 393 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0005_conf.xml: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741920

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0005_conf.xml: MISSING 1 blocks of total size 222876 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0006-1551595993100-hadoop-select+word%2C+count%281%29+c+from+%28select+...desc%28Stage-1551596007220-1-1-SUCCEEDED-root.hadoop-1551595996699.jhist: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741932

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0006-1551595993100-hadoop-select+word%2C+count%281%29+c+from+%28select+...desc%28Stage-1551596007220-1-1-SUCCEEDED-root.hadoop-1551595996699.jhist: MISSING 1 blocks of total size 35468 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0006.summary: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741931

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0006.summary: MISSING 1 blocks of total size 393 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0006_conf.xml: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741933

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0006_conf.xml: MISSING 1 blocks of total size 223246 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0007-1551596514844-hadoop-select+word%2C+count%281%29+c+from+%28select+...desc%28Stage-1551596530603-1-1-SUCCEEDED-root.hadoop-1551596519431.jhist: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741946

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0007-1551596514844-hadoop-select+word%2C+count%281%29+c+from+%28select+...desc%28Stage-1551596530603-1-1-SUCCEEDED-root.hadoop-1551596519431.jhist: MISSING 1 blocks of total size 35387 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0007.summary: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741945

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0007.summary: MISSING 1 blocks of total size 393 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0007_conf.xml: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741947

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0007_conf.xml: MISSING 1 blocks of total size 222877 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0008-1551596533679-hadoop-select+word%2C+count%281%29+c+from+%28select+...desc%28Stage-1551596548178-1-1-SUCCEEDED-root.hadoop-1551596537840.jhist: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741959

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0008-1551596533679-hadoop-select+word%2C+count%281%29+c+from+%28select+...desc%28Stage-1551596548178-1-1-SUCCEEDED-root.hadoop-1551596537840.jhist: MISSING 1 blocks of total size 35452 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0008.summary: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741958

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0008.summary: MISSING 1 blocks of total size 393 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0008_conf.xml: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741960

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0008_conf.xml: MISSING 1 blocks of total size 223247 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1552354714580_0001-1552357884539-hadoop-create+table+result+as+select+time%2C+spl...10%28Stage-1552357903031-1-1-SUCCEEDED-root.hadoop-1552357890795.jhist: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742003

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1552354714580_0001-1552357884539-hadoop-create+table+result+as+select+time%2C+spl...10%28Stage-1552357903031-1-1-SUCCEEDED-root.hadoop-1552357890795.jhist: MISSING 1 blocks of total size 35622 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1552354714580_0001.summary: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742002

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1552354714580_0001.summary: MISSING 1 blocks of total size 393 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1552354714580_0001_conf.xml: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742004

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1552354714580_0001_conf.xml: MISSING 1 blocks of total size 223199 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1553496559957_0003-1553497835135-hadoop-g6%2Dhadoop%2D1.0.jar-1553497852695-1-1-SUCCEEDED-root.hadoop-1553497842646.jhist: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742018

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1553496559957_0003-1553497835135-hadoop-g6%2Dhadoop%2D1.0.jar-1553497852695-1-1-SUCCEEDED-root.hadoop-1553497842646.jhist: MISSING 1 blocks of total size 33325 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1553496559957_0003.summary: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742017

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1553496559957_0003.summary: MISSING 1 blocks of total size 356 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1553496559957_0003_conf.xml: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742019

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1553496559957_0003_conf.xml: MISSING 1 blocks of total size 109348 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1553580778554_0001-1553591035008-hadoop-g6%2Dhadoop%2D1.0.jar-1553591047242-0-1-SUCCEEDED-root.hadoop-1553591041690.jhist: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742028

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1553580778554_0001-1553591035008-hadoop-g6%2Dhadoop%2D1.0.jar-1553591047242-0-1-SUCCEEDED-root.hadoop-1553591041690.jhist: MISSING 1 blocks of total size 23304 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1553580778554_0001.summary: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742027

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1553580778554_0001.summary: MISSING 1 blocks of total size 341 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1553580778554_0001_conf.xml: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742029

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1553580778554_0001_conf.xml: MISSING 1 blocks of total size 109348 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1553592774319_0002-1553592835033-hadoop-g6%2Dhadoop%2D1.0.jar-1553592854766-1-1-SUCCEEDED-root.hadoop-1553592840382.jhist: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742084

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1553592774319_0002-1553592835033-hadoop-g6%2Dhadoop%2D1.0.jar-1553592854766-1-1-SUCCEEDED-root.hadoop-1553592840382.jhist: MISSING 1 blocks of total size 33637 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1553592774319_0002.summary: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742083

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1553592774319_0002.summary: MISSING 1 blocks of total size 356 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1553592774319_0002_conf.xml: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742085

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1553592774319_0002_conf.xml: MISSING 1 blocks of total size 109348 B..
/user/hive/warehouse/d6_wc/ruozeinput.txt: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741934

/user/hive/warehouse/d6_wc/ruozeinput.txt: MISSING 1 blocks of total size 44 B..
/user/hive/warehouse/login/day=2019-03-11/login.log: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742005

/user/hive/warehouse/login/day=2019-03-11/login.log: MISSING 1 blocks of total size 1065422 B..
/user/hive/warehouse/result/000000_0: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742000

/user/hive/warehouse/result/000000_0: MISSING 1 blocks of total size 274 B..
/user/hive/warehouse/user_login/day=2019-03-11/login.log: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741990

/user/hive/warehouse/user_login/day=2019-03-11/login.log: MISSING 1 blocks of total size 159019 B..
/wordcount/input/a.log: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741842

/wordcount/input/a.log: MISSING 1 blocks of total size 75 B..
/wordcount/input/b.txt: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741843

/wordcount/input/b.txt: MISSING 1 blocks of total size 24 B...
/wordcount/output1/part-r-00000: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741851

/wordcount/output1/part-r-00000: MISSING 1 blocks of total size 105 B.Status: CORRUPT
 Total size:	57048243 B
 Total dirs:	50
 Total files:	64
 Total symlinks:		0
 Total blocks (validated):	62 (avg. block size 920132 B)
  ********************************
  CORRUPT FILES:	62
  MISSING BLOCKS:	62
  MISSING SIZE:		57048243 B
  CORRUPT BLOCKS: 	62
  ********************************
 Minimally replicated blocks:	0 (0.0 %)
 Over-replicated blocks:	0 (0.0 %)
 Under-replicated blocks:	0 (0.0 %)
 Mis-replicated blocks:		0 (0.0 %)
 Default replication factor:	1
 Average block replication:	0.0
 Corrupt blocks:		62
 Missing replicas:		0
 Number of data-nodes:		1
 Number of racks:		1
FSCK ended at Sat Apr 06 11:40:45 CST 2019 in 34 milliseconds


The filesystem under path '/' is CORRUPT

[hadoop@hadoop001 hadoop-2.6.0-cdh5.7.0]$ hdfs debug recoverLease -path /wordcount/output1/part-r-00000 -retries 10
Retry #9
recoverLease got exception:
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.SafeModeException): Cannot recover the lease of /wordcount/output1/part-r-00000. Name node is in safe mode.
The reported blocks 0 needs additional 62 blocks to reach the threshold 0.9990 of total blocks 62.
The number of live datanodes 1 has reached the minimum number 0. Safe mode will be turned off automatically once the thresholds have been reached.
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkNameNodeSafeMode(FSNamesystem.java:1418)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.recoverLease(FSNamesystem.java:3033)
	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.recoverLease(NameNodeRpcServer.java:621)
	at org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.recoverLease(AuthorizationProviderProxyClientProtocol.java:368)
	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.recoverLease(ClientNamenodeProtocolServerSideTranslatorPB.java:644)
	at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2086)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2082)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2080)

	at org.apache.hadoop.ipc.Client.call(Client.java:1471)
	at org.apache.hadoop.ipc.Client.call(Client.java:1408)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:230)
	at com.sun.proxy.$Proxy9.recoverLease(Unknown Source)
	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.recoverLease(ClientNamenodeProtocolTranslatorPB.java:589)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.in
)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:104)
	at com.sun.proxy.$Proxy10.recoverLease(Unknown Source)
	at org.apache.hadoop.hdfs.DFSClient.recoverLease(DFSClient.java:1278)
	at org.apache.hadoop.hdfs.DistributedFileSystem$2.doCall(DistributedFileSystem.java:285)
	at org.apache.hadoop.hdfs.DistributedFileSystem$2.doCall(DistributedFileSystem.java:281)
	at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
	at org.apache.hadoop.hdfs.DistributedFileSystem.recoverLease(DistributedFileSystem.java:281)
	at org.apache.hadoop.hdfs.tools.DebugAdmin$RecoverLeaseCommand.run(DebugAdmin.java:256)
	at org.apache.hadoop.hdfs.tools.DebugAdmin.run(DebugAdmin.java:336)
	at org.apache.hadoop.hdfs.tools.DebugAdmin.main(DebugAdmin.java:359)
Giving up on recoverLease for /wordcount/output1/part-r-00000 after 10 tries.

 

[hadoop@hadoop001 hadoop-2.6.0-cdh5.7.0]$ hdfs dfsadmin -safemode leave
19/04/06 12:09:11 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Safe mode is OFF

 

[hadoop@hadoop001 hadoop-2.6.0-cdh5.7.0]$ hdfs debug recoverLease -path /wordcount/output1/part-r-00000 -retries 10
19/04/06 12:09:28 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
recoverLease SUCCEEDED on /wordcount/output1/part-r-00000

但其实因为我只有一个节点，其实块损坏了，数据也就丢失了，所以其实并没有修复成功

下面换到集群的模式进行测试

[hadoop@hadoop001 ~]$ hdfs fsck /
19/04/07 18:31:03 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Connecting to namenode via http://hadoop002:50070
FSCK started by hadoop (auth:SIMPLE) from /172.18.32.193 for path / at Sun Apr 07 18:31:04 CST 2019
......Status: HEALTHY
 Total size:	447146 B
 Total dirs:	20
 Total files:	6
 Total symlinks:		0
 Total blocks (validated):	6 (avg. block size 74524 B)
 Minimally replicated blocks:	6 (100.0 %)
 Over-replicated blocks:	0 (0.0 %)
 Under-replicated blocks:	0 (0.0 %)
 Mis-replicated blocks:		0 (0.0 %)
 Default replication factor:	3
 Average block replication:	3.0
 Corrupt blocks:		0
 Missing replicas:		0 (0.0 %)
 Number of data-nodes:		3
 Number of racks:		1
FSCK ended at Sun Apr 07 18:31:04 CST 2019 in 10 milliseconds


The filesystem under path '/' is HEALTHY
[hadoop@hadoop001 ~]$ hdfs dfs -mkdir /blockrecover
19/04/07 18:33:38 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable


[hadoop@hadoop001 ~]$ hdfs dfs -ls /
19/04/07 18:33:47 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Found 4 items
-rw-r--r--   3 hadoop hadoop        238 2019-04-02 00:38 /README.txt
drwxr-xr-x   - hadoop hadoop          0 2019-04-07 18:33 /blockrecover
drwxrwx---   - hadoop hadoop          0 2019-04-02 00:53 /tmp
drwxr-xr-x   - hadoop hadoop          0 2019-04-02 00:53 /user
[hadoop@hadoop001 data]$ hdfs dfs -put ruozedata.md /blockrecover
19/04/07 18:36:01 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

[hadoop@hadoop001 data]$ hdfs dfs -ls /blockrecover/ruozedata.md
19/04/07 18:36:32 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
-rw-r--r--   3 hadoop hadoop         18 2019-04-07 18:36 /blockrecover/ruozedata.md

[hadoop@hadoop001 data]$ hdfs dfs -text /blockrecover/ruozedata.md
19/04/07 18:36:47 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
www.ruozedata.com
[hadoop@hadoop001 data]$ hdfs fsck /
19/04/07 18:37:16 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Connecting to namenode via http://hadoop002:50070
FSCK started by hadoop (auth:SIMPLE) from /172.18.32.193 for path / at Sun Apr 07 18:37:17 CST 2019
.......Status: HEALTHY
 Total size:	447164 B
 Total dirs:	21
 Total files:	7
 Total symlinks:		0
 Total blocks (validated):	7 (avg. block size 63880 B)
 Minimally replicated blocks:	7 (100.0 %)
 Over-replicated blocks:	0 (0.0 %)
 Under-replicated blocks:	0 (0.0 %)
 Mis-replicated blocks:		0 (0.0 %)
 Default replication factor:	3
 Average block replication:	3.0
 Corrupt blocks:		0
 Missing replicas:		0 (0.0 %)
 Number of data-nodes:		3
 Number of racks:		1
FSCK ended at Sun Apr 07 18:37:17 CST 2019 in 4 milliseconds


The filesystem under path '/' is HEALTHY

 停止Hadoop集群

[hadoop@hadoop001 subdir0]$ stop-dfs.sh
19/04/07 20:14:33 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Stopping namenodes on [hadoop001 hadoop002]
hadoop001: stopping namenode
hadoop002: stopping namenode
hadoop003: stopping datanode
hadoop001: stopping datanode
hadoop002: stopping datanode
Stopping journal nodes [hadoop001 hadoop002 hadoop003]
hadoop001: stopping journalnode
hadoop003: stopping journalnode
hadoop002: stopping journalnode
19/04/07 20:14:51 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Stopping ZK Failover Controllers on NN hosts [hadoop001 hadoop002]
hadoop001: stopping zkfc
hadoop002: stopping zkfc

[hadoop@hadoop001 subdir0]$ jps
1616 QuorumPeerMain
5810 Jps
2597 ResourceManager
3158 JobHistoryServer
2698 NodeManager

启动Hadoop集群

[hadoop@hadoop001 subdir0]$ start-dfs.sh
19/04/07 20:15:51 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Starting namenodes on [hadoop001 hadoop002]
hadoop002: starting namenode, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.7.0/logs/hadoop-hadoop-namenode-hadoop002.out
hadoop001: starting namenode, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.7.0/logs/hadoop-hadoop-namenode-hadoop001.out
hadoop002: starting datanode, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.7.0/logs/hadoop-hadoop-datanode-hadoop002.out
hadoop001: starting datanode, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.7.0/logs/hadoop-hadoop-datanode-hadoop001.out
hadoop003: starting datanode, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.7.0/logs/hadoop-hadoop-datanode-hadoop003.out
Starting journal nodes [hadoop001 hadoop002 hadoop003]
hadoop001: starting journalnode, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.7.0/logs/hadoop-hadoop-journalnode-hadoop001.out
hadoop002: starting journalnode, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.7.0/logs/hadoop-hadoop-journalnode-hadoop002.out
hadoop003: starting journalnode, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.7.0/logs/hadoop-hadoop-journalnode-hadoop003.out
19/04/07 20:16:07 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Starting ZK Failover Controllers on NN hosts [hadoop001 hadoop002]
hadoop002: starting zkfc, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.7.0/logs/hadoop-hadoop-zkfc-hadoop002.out
hadoop001: starting zkfc, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.7.0/logs/hadoop-hadoop-zkfc-hadoop001.out


[hadoop@hadoop001 subdir0]$ jps
1616 QuorumPeerMain
6403 DFSZKFailoverController
5924 NameNode
2597 ResourceManager
3158 JobHistoryServer
6455 Jps
6216 JournalNode
2698 NodeManager
6027 DataNode
[hadoop@hadoop001 subdir0]$ hdfs fsck /
19/04/07 20:16:47 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Connecting to namenode via http://hadoop002:50070
FSCK started by hadoop (auth:SIMPLE) from /172.18.32.193 for path / at Sun Apr 07 20:16:48 CST 2019
.......Status: HEALTHY
 Total size:	447164 B
 Total dirs:	21
 Total files:	7
 Total symlinks:		0
 Total blocks (validated):	7 (avg. block size 63880 B)
 Minimally replicated blocks:	7 (100.0 %)
 Over-replicated blocks:	0 (0.0 %)
 Under-replicated blocks:	0 (0.0 %)
 Mis-replicated blocks:		0 (0.0 %)
 Default replication factor:	3
 Average block replication:	3.0
 Corrupt blocks:		0
 Missing replicas:		0 (0.0 %)
 Number of data-nodes:		3
 Number of racks:		1
FSCK ended at Sun Apr 07 20:16:48 CST 2019 in 22 milliseconds


The filesystem under path '/' is HEALTHY
[hadoop@hadoop001 subdir0]$ ls
blk_1073741825            blk_1073741840_1016.meta  blk_1073741843
blk_1073741825_1001.meta  blk_1073741841            blk_1073741843_1019.meta
blk_1073741839            blk_1073741841_1017.meta  blk_1073741844
blk_1073741839_1015.meta  blk_1073741842            blk_1073741844_1020.meta
blk_1073741840            blk_1073741842_1018.meta

太快了，没看到预想结果，来一个文件大点的

[hadoop@hadoop001 data]$ hdfs dfs -put banking.txt /blockrecover
19/04/07 21:47:51 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

[hadoop@hadoop001 data]$ hdfs dfs -ls /blockrecover
19/04/07 21:48:19 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Found 2 items
-rw-r--r--   3 hadoop hadoop  483671526 2019-04-07 21:48 /blockrecover/banking.txt
-rw-r--r--   3 hadoop hadoop         18 2019-04-07 18:36 /blockrecover/ruozedata.md


[hadoop@hadoop001 data]$ cd /home/hadoop/app/hadoop/data/dfs/data/current/BP-1152973221-172.18.32.193-1554134553451/current/finalized/subdir0/subdir0

[hadoop@hadoop001 subdir0]$ ls
blk_1073741825            blk_1073741841            blk_1073741844
blk_1073741825_1001.meta  blk_1073741841_1017.meta  blk_1073741844_1020.meta
blk_1073741839            blk_1073741842            blk_1073741845
blk_1073741839_1015.meta  blk_1073741842_1018.meta  blk_1073741845_1021.meta
blk_1073741840            blk_1073741843            blk_1073741846
blk_1073741840_1016.meta  blk_1073741843_1019.meta  blk_1073741846_1022.meta

[hadoop@hadoop001 subdir0]$ rm -fr blk_1073741845 blk_1073741845_1021.meta

[hadoop@hadoop001 subdir0]$ ls
blk_1073741825            blk_1073741841            blk_1073741844
blk_1073741825_1001.meta  blk_1073741841_1017.meta  blk_1073741844_1020.meta
blk_1073741839            blk_1073741842            blk_1073741846
blk_1073741839_1015.meta  blk_1073741842_1018.meta  blk_1073741846_1022.meta
blk_1073741840            blk_1073741843
blk_1073741840_1016.meta  blk_1073741843_1019.meta

停止Hadoop模拟数据丢失

[hadoop@hadoop001 subdir0]$ stop-dfs.sh

[hadoop@hadoop001 subdir0]$ start-dfs.sh

查看结果

[hadoop@hadoop001 subdir0]$ hdfs fsck /
19/04/07 21:51:35 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Connecting to namenode via http://hadoop002:50070
FSCK started by hadoop (auth:SIMPLE) from /172.18.32.193 for path / at Sun Apr 07 21:51:36 CST 2019
..
/blockrecover/banking.txt:  Under replicated BP-1152973221-172.18.32.193-1554134553451:blk_1073741845_1021. Target Replicas is 3 but found 2 replica(s).
......Status: HEALTHY
 Total size:	484118690 B
 Total dirs:	21
 Total files:	8
 Total symlinks:		0
 Total blocks (validated):	9 (avg. block size 53790965 B)
 Minimally replicated blocks:	9 (100.0 %)
 Over-replicated blocks:	0 (0.0 %)
 Under-replicated blocks:	1 (11.111111 %)
 Mis-replicated blocks:		0 (0.0 %)
 Default replication factor:	3
 Average block replication:	2.8888888
 Corrupt blocks:		0
 Missing replicas:		1 (3.7037036 %)
 Number of data-nodes:		3
 Number of racks:		1
FSCK ended at Sun Apr 07 21:51:36 CST 2019 in 15 milliseconds


The filesystem under path '/' is HEALTHY

过了几分钟，数据修复了

[hadoop@hadoop001 subdir0]$ hdfs fsck /
19/04/07 21:55:25 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Connecting to namenode via http://hadoop002:50070
FSCK started by hadoop (auth:SIMPLE) from /172.18.32.193 for path / at Sun Apr 07 21:55:26 CST 2019
........Status: HEALTHY
 Total size:	484118690 B
 Total dirs:	21
 Total files:	8
 Total symlinks:		0
 Total blocks (validated):	9 (avg. block size 53790965 B)
 Minimally replicated blocks:	9 (100.0 %)
 Over-replicated blocks:	0 (0.0 %)
 Under-replicated blocks:	0 (0.0 %)
 Mis-replicated blocks:		0 (0.0 %)
 Default replication factor:	3
 Average block replication:	3.0
 Corrupt blocks:		0
 Missing replicas:		0 (0.0 %)
 Number of data-nodes:		3
 Number of racks:		1
FSCK ended at Sun Apr 07 21:55:26 CST 2019 in 5 milliseconds


The filesystem under path '/' is HEALTHY

再来一次，这次手动来修复

[hadoop@hadoop001 subdir0]$ ls
blk_1073741825            blk_1073741841            blk_1073741844
blk_1073741825_1001.meta  blk_1073741841_1017.meta  blk_1073741844_1020.meta
blk_1073741839            blk_1073741842            blk_1073741845
blk_1073741839_1015.meta  blk_1073741842_1018.meta  blk_1073741845_1021.meta
blk_1073741840            blk_1073741843            blk_1073741846
blk_1073741840_1016.meta  blk_1073741843_1019.meta  blk_1073741846_1022.meta

[hadoop@hadoop001 subdir0]$ rm -fr blk_1073741845 blk_1073741845_1021.meta

[hadoop@hadoop001 subdir0]$ ls
blk_1073741825            blk_1073741841            blk_1073741844
blk_1073741825_1001.meta  blk_1073741841_1017.meta  blk_1073741844_1020.meta
blk_1073741839            blk_1073741842            blk_1073741846
blk_1073741839_1015.meta  blk_1073741842_1018.meta  blk_1073741846_1022.meta
blk_1073741840            blk_1073741843
blk_1073741840_1016.meta  blk_1073741843_1019.meta
[hadoop@hadoop001 subdir0]$ stop-dfs.sh

[hadoop@hadoop001 subdir0]$ start-dfs.sh

重启以后，可以看到有一个副本丢失，而且ls可以看到删除的文件还没有被自动恢复 

[hadoop@hadoop001 subdir0]$ hdfs fsck /
19/04/07 22:07:45 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Connecting to namenode via http://hadoop002:50070
FSCK started by hadoop (auth:SIMPLE) from /172.18.32.193 for path / at Sun Apr 07 22:07:46 CST 2019
..
/blockrecover/banking.txt:  Under replicated BP-1152973221-172.18.32.193-1554134553451:blk_1073741845_1021. Target Replicas is 3 but found 2 replica(s).
......Status: HEALTHY
 Total size:	484118690 B
 Total dirs:	21
 Total files:	8
 Total symlinks:		0
 Total blocks (validated):	9 (avg. block size 53790965 B)
 Minimally replicated blocks:	9 (100.0 %)
 Over-replicated blocks:	0 (0.0 %)
 Under-replicated blocks:	1 (11.111111 %)
 Mis-replicated blocks:		0 (0.0 %)
 Default replication factor:	3
 Average block replication:	2.8888888
 Corrupt blocks:		0
 Missing replicas:		1 (3.7037036 %)
 Number of data-nodes:		3
 Number of racks:		1
FSCK ended at Sun Apr 07 22:07:46 CST 2019 in 108 milliseconds


The filesystem under path '/' is HEALTHY

 

[hadoop@hadoop001 subdir0]$ ls
blk_1073741825            blk_1073741841            blk_1073741844
blk_1073741825_1001.meta  blk_1073741841_1017.meta  blk_1073741844_1020.meta
blk_1073741839            blk_1073741842            blk_1073741846
blk_1073741839_1015.meta  blk_1073741842_1018.meta  blk_1073741846_1022.meta
blk_1073741840            blk_1073741843
blk_1073741840_1016.meta  blk_1073741843_1019.meta

手动修复 

[hadoop@hadoop001 subdir0]$ hdfs debug recoverLease -path /blockrecover/banking.txt -retries 10
19/04/07 22:07:58 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

recoverLease SUCCEEDED on /blockrecover/banking.txt

成功了 

[hadoop@hadoop001 subdir0]$ hdfs fsck /
19/04/07 22:08:16 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Connecting to namenode via http://hadoop002:50070
FSCK started by hadoop (auth:SIMPLE) from /172.18.32.193 for path / at Sun Apr 07 22:08:17 CST 2019
........Status: HEALTHY
 Total size:	484118690 B
 Total dirs:	21
 Total files:	8
 Total symlinks:		0
 Total blocks (validated):	9 (avg. block size 53790965 B)
 Minimally replicated blocks:	9 (100.0 %)
 Over-replicated blocks:	0 (0.0 %)
 Under-replicated blocks:	0 (0.0 %)
 Mis-replicated blocks:		0 (0.0 %)
 Default replication factor:	3
 Average block replication:	3.0
 Corrupt blocks:		0
 Missing replicas:		0 (0.0 %)
 Number of data-nodes:		3
 Number of racks:		1
FSCK ended at Sun Apr 07 22:08:17 CST 2019 in 5 milliseconds


The filesystem under path '/' is HEALTHY
[hadoop@hadoop001 subdir0]$ ls
blk_1073741825            blk_1073741841            blk_1073741844
blk_1073741825_1001.meta  blk_1073741841_1017.meta  blk_1073741844_1020.meta
blk_1073741839            blk_1073741842            blk_1073741845
blk_1073741839_1015.meta  blk_1073741842_1018.meta  blk_1073741845_1021.meta
blk_1073741840            blk_1073741843            blk_1073741846
blk_1073741840_1016.meta  blk_1073741843_1019.meta  blk_1073741846_1022.meta

 

AaronLwx
关注
  • 0
    点赞
  • 踩
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
【HDFS篇08】NameNode故障处理1
08-04
总结来说，处理NameNode故障的关键在于利用Secondary NameNode的检查点数据恢复NameNode的状态。然而，这种手动恢复方法并不适合大规模生产环境，因此建议使用HA配置或者定期备份元数据，以提供更高级别的数据保护。...
2021SC@SDUSC hadoop源码分析（八）
qq_53102935的博客
11-15 959
2021SC@SDUSC 本篇我将对hadoop-tools中的Rumen进行分析 Rumen 概观 Rumen是一个数据提取和分析工具Apache Hadoop。Rumenmines工作历史日志提取有意义的数据，并以易于解析、压缩的格式或摘要。来自MapReduce日志的原始跟踪数据通常不足以进行模拟、仿真和基准测试，因为这些工具通常试图测量源数据中没有出现的情况。例如，如果一个任务在原始跟踪数据中本地运行，但调度程序的模拟选择在远程机架上运行该任务，模拟器需要一个其输入无法提供的运行时。为了..
2016-11-3 IDEA调试hadoop的错误
程序萌部落
11-03 794
标签： 有道云笔记Cannot delete /tmp/hadoop-yarn/staging/hadoop/.staging/job_1477796535608_0001. Name node is in safe mode. Linux集群中的namenode没有关闭safemode 2016-11-01 18:32:27,979 INFO [main] mapred.ClientServ
hadoop常见错误及解决办法！
热门推荐
沉底的石头
05-30 1万+
1：Shuffle Error: Exceeded MAX_FAILED_UNIQUE_FETCHES; bailing-out  Answer： 程序里面需要打开多个文件，进行分析，系统一般默认数量是1024，（用ulimit -a可以看到）对于正常使用是够了，但是对于程序来讲，就太少了。 修改办法： 修改2个文件。        /etc/security/limits.
hive报错Could not get block locations. Source file “/user/hive/warehouse/xxx
博客
12-24 3361
hive报错 Could not get block locations. Source file "/user/hive/warehouse/xxx... 以及 Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Error while processing row (tag=0)... 是因为链接超时了设置set mapred.task.timeout=600000;为10分钟或更长即可 ...
Hadoop采坑小记
Source_00的博客
10-20 1921
一 eclipse 上传文件到hdfs报错： java.io.IOException: Could not get block locations. Source file "/1108/daxian/banzhang/kafka_2.12-2.0.0.tgz" - Aborting... at org.apache.hadoop.hdfs.DataStreamer.setupPipeline...
HDFS详解②
01-20
【HDFS详解②】 Hadoop分布式文件系统（HDFS）是大数据处理的核心组件，它提供了高容错性、可扩展性和高效的数据访问能力。本文将深入探讨HDFS的两个关键方面...理解这些核心概念对于优化HDFS性能和解决故障至关重要。
HDFS简介
01-07
一、HDFS的简介 分布式文件系统 HDFS Hadoop Distributed FileSystem 基于流数据访问模式处理超大规模的文件 适合应用大规模的数据集上 HDFS的优点 1）处理超大规模的文件 ...检测和快速恢复硬件故障
Hadoop HDFS初级部分
最新发布
04-21
- **高容错性**：能够容忍节点故障，自动恢复数据丢失。 - **高吞吐量**：支持大规模数据集的快速访问。 - **成本效益**：可以在廉价的商品硬件上运行。 #### 二、HDFS架构设计 **HDFS架构**主要包括三个核心组件...
HDFS综述1
08-03
1. **高度容错性**：HDFS设计的核心目标之一是应对硬件故障。由于文件系统可能由数百甚至数千台服务器组成，每台服务器都存储部分数据，因此硬件失效是常态。HDFS通过错误检测和快速自动恢复机制确保数据的可靠性。 ...
【1】HDFS概述
01-20
- 高容错性：通过数据冗余，确保即使部分节点故障，数据也能被恢复。 - 扩展性强：可以轻松添加更多硬件资源来扩大存储和处理能力。 - 高性能：适合大规模并行处理，适合大数据的批量读取。 （2）缺点： - 不...
HDFS文件系统.pdf
06-27
在设计思路上，HDFS假设硬件故障是常态而非异常，因此其核心设计目标是能够快速检测和自动恢复错误。系统由大量的数据节点（DataNodes）组成，每个节点存储部分文件数据，确保即使有节点故障，数据仍然可以被访问。...
hdfs优缺点
06-04
2. **快速数据恢复**：HDFS的设计使其能够在检测到节点故障时自动进行数据恢复操作，这极大地提高了系统的可用性。 #### 支持大数据处理框架 1. **与Hadoop的紧密集成**：HDFS作为Hadoop的核心组件之一，与Hadoop ...
大数据HDFS文档
08-23
- 允许管理员进行各种维护操作，如检查磁盘空间使用情况、修复损坏的文件等。 #### HDFS文件存储类型 - **行式文件**: - 通常用于存储文本数据或CSV文件等。 - 易于读写，但不太适合进行随机访问或复杂的数据...
HDFS-Explore
11-21
HDFS具有元数据管理、数据块复制、故障检测和恢复等功能，使得大规模数据处理变得可能。 HDFS-Explore的特色功能包括： 1. **文件和文件夹的操作**：用户可以像在本地文件系统一样创建、删除、重命名HDFS中的文件...
Hadoop的jobhistoryserver配置
过河卒子Rover的博客
01-19 1万+
JobHistoryServer配置
CDH 丢失块及副本不足的块
只是甲的博客
03-11 3449
备注: CDH 6.3.1 文章目录一.问题描述二.解决方案2.1 丢失块查找2.2 解决oozie副本块不足的问题2.2.1 设置3个副本模式2.2.2 删除坏的块:参考 一.问题描述 如下图所示，刚安装的CDH,提示存在丢失块，也存在副本不足的块 二.解决方案 2.1 丢失块查找 sudo -u hdfs hadoop fsck / -files -blocks 测试记录: FSCK started by hdfs (auth:SIMPLE) from /10.31.1.123 for path /
Hadoop3 Safemode
liuwei0376的专栏
03-26 717
集群环境: CHD: 6.1 Environment : MYTEST OS : CentOS Linux release 7.7.1908 (Core) ROLE: 1NN, 5DN 问题症状 打开CM, 发现hdfs组件飘红, 点进去后发现集群处于safemode状态. 于是,打开dfshealth健康检查页面(注意hadoop3之前为50070, hadoop3之后为9870)查看后, ...
Hadoop原理详解：HDFS故障恢复与MapReduce实战
Hadoop的核心组成部分包括分布式文件系统 Hadoop Distributed File System (HDFS) 和 MapReduce 用于处理大规模数据处理任务。 HDFS的设计原则基于硬件错误的常态，它采用简单的一次写多次读一致性模型，适用于...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值