Hdfs 故障恢复

最新推荐文章于 2022-05-23 11:34:25 发布
最新推荐文章于 2022-05-23 11:34:25 发布
阅读量1k 收藏
点赞数
分类专栏： Hadoop
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/xiaoxiongaa0/article/details/89053695
版权

 下面以伪分布式为例来进行故障恢复。ok，let's go

[hadoop@hadoop001 hadoop-2.6.0-cdh5.7.0]$ hdfs fsck /
19/04/06 11:40:44 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Connecting to namenode via http://hadoop001:50070
FSCK started by hadoop (auth:SIMPLE) from /172.18.32.186 for path / at Sat Apr 06 11:40:45 CST 2019
.
/g6/hadoop/access/clear/day=20180717/part-r-00000: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742082

/g6/hadoop/access/clear/day=20180717/part-r-00000: MISSING 1 blocks of total size 9116696 B...
/g6/hadoop/accesslog/20180717/test.log: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742070

/g6/hadoop/accesslog/20180717/test.log: MISSING 1 blocks of total size 43282330 B..
/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0002/job.jar: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742063

/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0002/job.jar: MISSING 1 blocks of total size 6732 B..
/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0002/job.split: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742064

/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0002/job.split: MISSING 1 blocks of total size 4099 B..
/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0002/job.splitmetainfo: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742065

/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0002/job.splitmetainfo: MISSING 1 blocks of total size 633 B..
/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0002/job.xml: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742066

/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0002/job.xml: MISSING 1 blocks of total size 91915 B..
/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0002/job_1553580778554_0002.summary: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742068

/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0002/job_1553580778554_0002.summary: MISSING 1 blocks of total size 297 B..
/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0002/job_1553580778554_0002_1.jhist: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742069

/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0002/job_1553580778554_0002_1.jhist: MISSING 1 blocks of total size 42661 B..
/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0002/job_1553580778554_0002_1_conf.xml: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742067

/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0002/job_1553580778554_0002_1_conf.xml: MISSING 1 blocks of total size 109349 B..
/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0003/job.jar: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742071

/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0003/job.jar: MISSING 1 blocks of total size 6732 B..
/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0003/job.split: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742072

/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0003/job.split: MISSING 1 blocks of total size 131 B..
/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0003/job.splitmetainfo: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742073

/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0003/job.splitmetainfo: MISSING 1 blocks of total size 27 B..
/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0003/job.xml: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742074

/tmp/hadoop-yarn/staging/hadoop/.staging/job_1553580778554_0003/job.xml: MISSING 1 blocks of total size 91914 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1550983677568_0001-1550984046181-hadoop-QuasiMonteCarlo-1550984067738-5-1-SUCCEEDED-root.hadoop-1550984051208.jhist: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741840

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1550983677568_0001-1550984046181-hadoop-QuasiMonteCarlo-1550984067738-5-1-SUCCEEDED-root.hadoop-1550984051208.jhist: MISSING 1 blocks of total size 59168 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1550983677568_0001.summary: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741839

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1550983677568_0001.summary: MISSING 1 blocks of total size 355 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1550983677568_0001_conf.xml: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741841

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1550983677568_0001_conf.xml: MISSING 1 blocks of total size 110148 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1550983677568_0002-1550984832078-hadoop-word+count-1550984847506-2-1-SUCCEEDED-root.hadoop-1550984836666.jhist: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741853

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1550983677568_0002-1550984832078-hadoop-word+count-1550984847506-2-1-SUCCEEDED-root.hadoop-1550984836666.jhist: MISSING 1 blocks of total size 40208 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1550983677568_0002.summary: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741852

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1550983677568_0002.summary: MISSING 1 blocks of total size 349 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1550983677568_0002_conf.xml: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741854

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1550983677568_0002_conf.xml: MISSING 1 blocks of total size 109840 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0001-1551595710002-hadoop-select+word%2C+count%281%29+c%0Afrom%0A%28select+...desc%28Stage-1551595727139-1-1-SUCCEEDED-root.hadoop-1551595716018.jhist: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741867

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0001-1551595710002-hadoop-select+word%2C+count%281%29+c%0Afrom%0A%28select+...desc%28Stage-1551595727139-1-1-SUCCEEDED-root.hadoop-1551595716018.jhist: MISSING 1 blocks of total size 35410 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0001.summary: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741866

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0001.summary: MISSING 1 blocks of total size 395 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0001_conf.xml: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741868

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0001_conf.xml: MISSING 1 blocks of total size 222878 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0002-1551595730185-hadoop-select+word%2C+count%281%29+c%0Afrom%0A%28select+...desc%28Stage-1551595744690-1-1-SUCCEEDED-root.hadoop-1551595734416.jhist: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741880

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0002-1551595730185-hadoop-select+word%2C+count%281%29+c%0Afrom%0A%28select+...desc%28Stage-1551595744690-1-1-SUCCEEDED-root.hadoop-1551595734416.jhist: MISSING 1 blocks of total size 35475 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0002.summary: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741879

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0002.summary: MISSING 1 blocks of total size 395 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0002_conf.xml: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741881

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0002_conf.xml: MISSING 1 blocks of total size 223248 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0003-1551595813288-hadoop-select+word%2C+count%281%29+c+from+%28select+...desc%28Stage-1551595828436-1-1-SUCCEEDED-root.hadoop-1551595817375.jhist: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741893

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0003-1551595813288-hadoop-select+word%2C+count%281%29+c+from+%28select+...desc%28Stage-1551595828436-1-1-SUCCEEDED-root.hadoop-1551595817375.jhist: MISSING 1 blocks of total size 35403 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0003.summary: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741892

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0003.summary: MISSING 1 blocks of total size 393 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0003_conf.xml: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741894

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0003_conf.xml: MISSING 1 blocks of total size 222876 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0004-1551595830949-hadoop-select+word%2C+count%281%29+c+from+%28select+...desc%28Stage-1551595845881-1-1-SUCCEEDED-root.hadoop-1551595835253.jhist: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741906

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0004-1551595830949-hadoop-select+word%2C+count%281%29+c+from+%28select+...desc%28Stage-1551595845881-1-1-SUCCEEDED-root.hadoop-1551595835253.jhist: MISSING 1 blocks of total size 35479 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0004.summary: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741905

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0004.summary: MISSING 1 blocks of total size 393 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0004_conf.xml: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741907

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0004_conf.xml: MISSING 1 blocks of total size 223246 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0005-1551595975632-hadoop-select+word%2C+count%281%29+c+from+%28select+...desc%28Stage-1551595991154-1-1-SUCCEEDED-root.hadoop-1551595979967.jhist: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741919

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0005-1551595975632-hadoop-select+word%2C+count%281%29+c+from+%28select+...desc%28Stage-1551595991154-1-1-SUCCEEDED-root.hadoop-1551595979967.jhist: MISSING 1 blocks of total size 35403 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0005.summary: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741918

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0005.summary: MISSING 1 blocks of total size 393 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0005_conf.xml: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741920

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0005_conf.xml: MISSING 1 blocks of total size 222876 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0006-1551595993100-hadoop-select+word%2C+count%281%29+c+from+%28select+...desc%28Stage-1551596007220-1-1-SUCCEEDED-root.hadoop-1551595996699.jhist: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741932

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0006-1551595993100-hadoop-select+word%2C+count%281%29+c+from+%28select+...desc%28Stage-1551596007220-1-1-SUCCEEDED-root.hadoop-1551595996699.jhist: MISSING 1 blocks of total size 35468 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0006.summary: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741931

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0006.summary: MISSING 1 blocks of total size 393 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0006_conf.xml: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741933

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0006_conf.xml: MISSING 1 blocks of total size 223246 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0007-1551596514844-hadoop-select+word%2C+count%281%29+c+from+%28select+...desc%28Stage-1551596530603-1-1-SUCCEEDED-root.hadoop-1551596519431.jhist: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741946

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0007-1551596514844-hadoop-select+word%2C+count%281%29+c+from+%28select+...desc%28Stage-1551596530603-1-1-SUCCEEDED-root.hadoop-1551596519431.jhist: MISSING 1 blocks of total size 35387 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0007.summary: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741945

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0007.summary: MISSING 1 blocks of total size 393 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0007_conf.xml: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741947

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0007_conf.xml: MISSING 1 blocks of total size 222877 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0008-1551596533679-hadoop-select+word%2C+count%281%29+c+from+%28select+...desc%28Stage-1551596548178-1-1-SUCCEEDED-root.hadoop-1551596537840.jhist: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741959

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0008-1551596533679-hadoop-select+word%2C+count%281%29+c+from+%28select+...desc%28Stage-1551596548178-1-1-SUCCEEDED-root.hadoop-1551596537840.jhist: MISSING 1 blocks of total size 35452 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0008.summary: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741958

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0008.summary: MISSING 1 blocks of total size 393 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0008_conf.xml: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741960

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1551594909173_0008_conf.xml: MISSING 1 blocks of total size 223247 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1552354714580_0001-1552357884539-hadoop-create+table+result+as+select+time%2C+spl...10%28Stage-1552357903031-1-1-SUCCEEDED-root.hadoop-1552357890795.jhist: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742003

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1552354714580_0001-1552357884539-hadoop-create+table+result+as+select+time%2C+spl...10%28Stage-1552357903031-1-1-SUCCEEDED-root.hadoop-1552357890795.jhist: MISSING 1 blocks of total size 35622 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1552354714580_0001.summary: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742002

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1552354714580_0001.summary: MISSING 1 blocks of total size 393 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1552354714580_0001_conf.xml: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742004

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1552354714580_0001_conf.xml: MISSING 1 blocks of total size 223199 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1553496559957_0003-1553497835135-hadoop-g6%2Dhadoop%2D1.0.jar-1553497852695-1-1-SUCCEEDED-root.hadoop-1553497842646.jhist: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742018

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1553496559957_0003-1553497835135-hadoop-g6%2Dhadoop%2D1.0.jar-1553497852695-1-1-SUCCEEDED-root.hadoop-1553497842646.jhist: MISSING 1 blocks of total size 33325 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1553496559957_0003.summary: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742017

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1553496559957_0003.summary: MISSING 1 blocks of total size 356 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1553496559957_0003_conf.xml: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742019

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1553496559957_0003_conf.xml: MISSING 1 blocks of total size 109348 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1553580778554_0001-1553591035008-hadoop-g6%2Dhadoop%2D1.0.jar-1553591047242-0-1-SUCCEEDED-root.hadoop-1553591041690.jhist: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742028

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1553580778554_0001-1553591035008-hadoop-g6%2Dhadoop%2D1.0.jar-1553591047242-0-1-SUCCEEDED-root.hadoop-1553591041690.jhist: MISSING 1 blocks of total size 23304 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1553580778554_0001.summary: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742027

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1553580778554_0001.summary: MISSING 1 blocks of total size 341 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1553580778554_0001_conf.xml: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742029

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1553580778554_0001_conf.xml: MISSING 1 blocks of total size 109348 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1553592774319_0002-1553592835033-hadoop-g6%2Dhadoop%2D1.0.jar-1553592854766-1-1-SUCCEEDED-root.hadoop-1553592840382.jhist: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742084

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1553592774319_0002-1553592835033-hadoop-g6%2Dhadoop%2D1.0.jar-1553592854766-1-1-SUCCEEDED-root.hadoop-1553592840382.jhist: MISSING 1 blocks of total size 33637 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1553592774319_0002.summary: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742083

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1553592774319_0002.summary: MISSING 1 blocks of total size 356 B..
/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1553592774319_0002_conf.xml: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742085

/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1553592774319_0002_conf.xml: MISSING 1 blocks of total size 109348 B..
/user/hive/warehouse/d6_wc/ruozeinput.txt: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741934

/user/hive/warehouse/d6_wc/ruozeinput.txt: MISSING 1 blocks of total size 44 B..
/user/hive/warehouse/login/day=2019-03-11/login.log: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742005

/user/hive/warehouse/login/day=2019-03-11/login.log: MISSING 1 blocks of total size 1065422 B..
/user/hive/warehouse/result/000000_0: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073742000

/user/hive/warehouse/result/000000_0: MISSING 1 blocks of total size 274 B..
/user/hive/warehouse/user_login/day=2019-03-11/login.log: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741990

/user/hive/warehouse/user_login/day=2019-03-11/login.log: MISSING 1 blocks of total size 159019 B..
/wordcount/input/a.log: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741842

/wordcount/input/a.log: MISSING 1 blocks of total size 75 B..
/wordcount/input/b.txt: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741843

/wordcount/input/b.txt: MISSING 1 blocks of total size 24 B...
/wordcount/output1/part-r-00000: CORRUPT blockpool BP-312826295-120.79.193.132-1550919991536 block blk_1073741851

/wordcount/output1/part-r-00000: MISSING 1 blocks of total size 105 B.Status: CORRUPT
 Total size:	57048243 B
 Total dirs:	50
 Total files:	64
 Total symlinks:		0
 Total blocks (validated):	62 (avg. block size 920132 B)
  ********************************
  CORRUPT FILES:	62
  MISSING BLOCKS:	62
  MISSING SIZE:		57048243 B
  CORRUPT BLOCKS: 	62
  ********************************
 Minimally replicated blocks:	0 (0.0 %)
 Over-replicated blocks:	0 (0.0 %)
 Under-replicated blocks:	0 (0.0 %)
 Mis-replicated blocks:		0 (0.0 %)
 Default replication factor:	1
 Average block replication:	0.0
 Corrupt blocks:		62
 Missing replicas:		0
 Number of data-nodes:		1
 Number of racks:		1
FSCK ended at Sat Apr 06 11:40:45 CST 2019 in 34 milliseconds


The filesystem under path '/' is CORRUPT

[hadoop@hadoop001 hadoop-2.6.0-cdh5.7.0]$ hdfs debug recoverLease -path /wordcount/output1/part-r-00000 -retries 10
Retry #9
recoverLease got exception:
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.SafeModeException): Cannot recover the lease of /wordcount/output1/part-r-00000. Name node is in safe mode.
The reported blocks 0 needs additional 62 blocks to reach the threshold 0.9990 of total blocks 62.
The number of live datanodes 1 has reached the minimum number 0. Safe mode will be turned off automatically once the thresholds have been reached.
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkNameNodeSafeMode(FSNamesystem.java:1418)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.recoverLease(FSNamesystem.java:3033)
	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.recoverLease(NameNodeRpcServer.java:621)
	at org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.recoverLease(AuthorizationProviderProxyClientProtocol.java:368)
	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.recoverLease(ClientNamenodeProtocolServerSideTranslatorPB.java:644)
	at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2086)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2082)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2080)

	at org.apache.hadoop.ipc.Client.call(Client.java:1471)
	at org.apache.hadoop.ipc.Client.call(Client.java:1408)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:230)
	at com.sun.proxy.$Proxy9.recoverLease(Unknown Source)
	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.recoverLease(ClientNamenodeProtocolTranslatorPB.java:589)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.in
)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:104)
	at com.sun.proxy.$Proxy10.recoverLease(Unknown Source)
	at org.apache.hadoop.hdfs.DFSClient.recoverLease(DFSClient.java:1278)
	at org.apache.hadoop.hdfs.DistributedFileSystem$2.doCall(DistributedFileSystem.java:285)
	at org.apache.hadoop.hdfs.DistributedFileSystem$2.doCall(DistributedFileSystem.java:281)
	at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
	at org.apache.hadoop.hdfs.DistributedFileSystem.recoverLease(DistributedFileSystem.java:281)
	at org.apache.hadoop.hdfs.tools.DebugAdmin$RecoverLeaseCommand.run(DebugAdmin.java:256)
	at org.apache.hadoop.hdfs.tools.DebugAdmin.run(DebugAdmin.java:336)
	at org.apache.hadoop.hdfs.tools.DebugAdmin.main(DebugAdmin.java:359)
Giving up on recoverLease for /wordcount/output1/part-r-00000 after 10 tries.

 

[hadoop@hadoop001 hadoop-2.6.0-cdh5.7.0]$ hdfs dfsadmin -safemode leave
19/04/06 12:09:11 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Safe mode is OFF

 

[hadoop@hadoop001 hadoop-2.6.0-cdh5.7.0]$ hdfs debug recoverLease -path /wordcount/output1/part-r-00000 -retries 10
19/04/06 12:09:28 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
recoverLease SUCCEEDED on /wordcount/output1/part-r-00000

但其实因为我只有一个节点，其实块损坏了，数据也就丢失了，所以其实并没有修复成功

下面换到集群的模式进行测试

[hadoop@hadoop001 ~]$ hdfs fsck /
19/04/07 18:31:03 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Connecting to namenode via http://hadoop002:50070
FSCK started by hadoop (auth:SIMPLE) from /172.18.32.193 for path / at Sun Apr 07 18:31:04 CST 2019
......Status: HEALTHY
 Total size:	447146 B
 Total dirs:	20
 Total files:	6
 Total symlinks:		0
 Total blocks (validated):	6 (avg. block size 74524 B)
 Minimally replicated blocks:	6 (100.0 %)
 Over-replicated blocks:	0 (0.0 %)
 Under-replicated blocks:	0 (0.0 %)
 Mis-replicated blocks:		0 (0.0 %)
 Default replication factor:	3
 Average block replication:	3.0
 Corrupt blocks:		0
 Missing replicas:		0 (0.0 %)
 Number of data-nodes:		3
 Number of racks:		1
FSCK ended at Sun Apr 07 18:31:04 CST 2019 in 10 milliseconds


The filesystem under path '/' is HEALTHY
[hadoop@hadoop001 ~]$ hdfs dfs -mkdir /blockrecover
19/04/07 18:33:38 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable


[hadoop@hadoop001 ~]$ hdfs dfs -ls /
19/04/07 18:33:47 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Found 4 items
-rw-r--r--   3 hadoop hadoop        238 2019-04-02 00:38 /README.txt
drwxr-xr-x   - hadoop hadoop          0 2019-04-07 18:33 /blockrecover
drwxrwx---   - hadoop hadoop          0 2019-04-02 00:53 /tmp
drwxr-xr-x   - hadoop hadoop          0 2019-04-02 00:53 /user
[hadoop@hadoop001 data]$ hdfs dfs -put ruozedata.md /blockrecover
19/04/07 18:36:01 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

[hadoop@hadoop001 data]$ hdfs dfs -ls /blockrecover/ruozedata.md
19/04/07 18:36:32 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
-rw-r--r--   3 hadoop hadoop         18 2019-04-07 18:36 /blockrecover/ruozedata.md

[hadoop@hadoop001 data]$ hdfs dfs -text /blockrecover/ruozedata.md
19/04/07 18:36:47 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
www.ruozedata.com
[hadoop@hadoop001 data]$ hdfs fsck /
19/04/07 18:37:16 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Connecting to namenode via http://hadoop002:50070
FSCK started by hadoop (auth:SIMPLE) from /172.18.32.193 for path / at Sun Apr 07 18:37:17 CST 2019
.......Status: HEALTHY
 Total size:	447164 B
 Total dirs:	21
 Total files:	7
 Total symlinks:		0
 Total blocks (validated):	7 (avg. block size 63880 B)
 Minimally replicated blocks:	7 (100.0 %)
 Over-replicated blocks:	0 (0.0 %)
 Under-replicated blocks:	0 (0.0 %)
 Mis-replicated blocks:		0 (0.0 %)
 Default replication factor:	3
 Average block replication:	3.0
 Corrupt blocks:		0
 Missing replicas:		0 (0.0 %)
 Number of data-nodes:		3
 Number of racks:		1
FSCK ended at Sun Apr 07 18:37:17 CST 2019 in 4 milliseconds


The filesystem under path '/' is HEALTHY

 停止Hadoop集群

[hadoop@hadoop001 subdir0]$ stop-dfs.sh
19/04/07 20:14:33 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Stopping namenodes on [hadoop001 hadoop002]
hadoop001: stopping namenode
hadoop002: stopping namenode
hadoop003: stopping datanode
hadoop001: stopping datanode
hadoop002: stopping datanode
Stopping journal nodes [hadoop001 hadoop002 hadoop003]
hadoop001: stopping journalnode
hadoop003: stopping journalnode
hadoop002: stopping journalnode
19/04/07 20:14:51 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Stopping ZK Failover Controllers on NN hosts [hadoop001 hadoop002]
hadoop001: stopping zkfc
hadoop002: stopping zkfc

[hadoop@hadoop001 subdir0]$ jps
1616 QuorumPeerMain
5810 Jps
2597 ResourceManager
3158 JobHistoryServer
2698 NodeManager

启动Hadoop集群

[hadoop@hadoop001 subdir0]$ start-dfs.sh
19/04/07 20:15:51 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Starting namenodes on [hadoop001 hadoop002]
hadoop002: starting namenode, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.7.0/logs/hadoop-hadoop-namenode-hadoop002.out
hadoop001: starting namenode, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.7.0/logs/hadoop-hadoop-namenode-hadoop001.out
hadoop002: starting datanode, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.7.0/logs/hadoop-hadoop-datanode-hadoop002.out
hadoop001: starting datanode, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.7.0/logs/hadoop-hadoop-datanode-hadoop001.out
hadoop003: starting datanode, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.7.0/logs/hadoop-hadoop-datanode-hadoop003.out
Starting journal nodes [hadoop001 hadoop002 hadoop003]
hadoop001: starting journalnode, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.7.0/logs/hadoop-hadoop-journalnode-hadoop001.out
hadoop002: starting journalnode, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.7.0/logs/hadoop-hadoop-journalnode-hadoop002.out
hadoop003: starting journalnode, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.7.0/logs/hadoop-hadoop-journalnode-hadoop003.out
19/04/07 20:16:07 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Starting ZK Failover Controllers on NN hosts [hadoop001 hadoop002]
hadoop002: starting zkfc, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.7.0/logs/hadoop-hadoop-zkfc-hadoop002.out
hadoop001: starting zkfc, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.7.0/logs/hadoop-hadoop-zkfc-hadoop001.out


[hadoop@hadoop001 subdir0]$ jps
1616 QuorumPeerMain
6403 DFSZKFailoverController
5924 NameNode
2597 ResourceManager
3158 JobHistoryServer
6455 Jps
6216 JournalNode
2698 NodeManager
6027 DataNode
[hadoop@hadoop001 subdir0]$ hdfs fsck /
19/04/07 20:16:47 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Connecting to namenode via http://hadoop002:50070
FSCK started by hadoop (auth:SIMPLE) from /172.18.32.193 for path / at Sun Apr 07 20:16:48 CST 2019
.......Status: HEALTHY
 Total size:	447164 B
 Total dirs:	21
 Total files:	7
 Total symlinks:		0
 Total blocks (validated):	7 (avg. block size 63880 B)
 Minimally replicated blocks:	7 (100.0 %)
 Over-replicated blocks:	0 (0.0 %)
 Under-replicated blocks:	0 (0.0 %)
 Mis-replicated blocks:		0 (0.0 %)
 Default replication factor:	3
 Average block replication:	3.0
 Corrupt blocks:		0
 Missing replicas:		0 (0.0 %)
 Number of data-nodes:		3
 Number of racks:		1
FSCK ended at Sun Apr 07 20:16:48 CST 2019 in 22 milliseconds


The filesystem under path '/' is HEALTHY
[hadoop@hadoop001 subdir0]$ ls
blk_1073741825            blk_1073741840_1016.meta  blk_1073741843
blk_1073741825_1001.meta  blk_1073741841            blk_1073741843_1019.meta
blk_1073741839            blk_1073741841_1017.meta  blk_1073741844
blk_1073741839_1015.meta  blk_1073741842            blk_1073741844_1020.meta
blk_1073741840            blk_1073741842_1018.meta

太快了，没看到预想结果，来一个文件大点的

[hadoop@hadoop001 data]$ hdfs dfs -put banking.txt /blockrecover
19/04/07 21:47:51 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

[hadoop@hadoop001 data]$ hdfs dfs -ls /blockrecover
19/04/07 21:48:19 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Found 2 items
-rw-r--r--   3 hadoop hadoop  483671526 2019-04-07 21:48 /blockrecover/banking.txt
-rw-r--r--   3 hadoop hadoop         18 2019-04-07 18:36 /blockrecover/ruozedata.md


[hadoop@hadoop001 data]$ cd /home/hadoop/app/hadoop/data/dfs/data/current/BP-1152973221-172.18.32.193-1554134553451/current/finalized/subdir0/subdir0

[hadoop@hadoop001 subdir0]$ ls
blk_1073741825            blk_1073741841            blk_1073741844
blk_1073741825_1001.meta  blk_1073741841_1017.meta  blk_1073741844_1020.meta
blk_1073741839            blk_1073741842            blk_1073741845
blk_1073741839_1015.meta  blk_1073741842_1018.meta  blk_1073741845_1021.meta
blk_1073741840            blk_1073741843            blk_1073741846
blk_1073741840_1016.meta  blk_1073741843_1019.meta  blk_1073741846_1022.meta

[hadoop@hadoop001 subdir0]$ rm -fr blk_1073741845 blk_1073741845_1021.meta

[hadoop@hadoop001 subdir0]$ ls
blk_1073741825            blk_1073741841            blk_1073741844
blk_1073741825_1001.meta  blk_1073741841_1017.meta  blk_1073741844_1020.meta
blk_1073741839            blk_1073741842            blk_1073741846
blk_1073741839_1015.meta  blk_1073741842_1018.meta  blk_1073741846_1022.meta
blk_1073741840            blk_1073741843
blk_1073741840_1016.meta  blk_1073741843_1019.meta

停止Hadoop模拟数据丢失

[hadoop@hadoop001 subdir0]$ stop-dfs.sh

[hadoop@hadoop001 subdir0]$ start-dfs.sh

查看结果

[hadoop@hadoop001 subdir0]$ hdfs fsck /
19/04/07 21:51:35 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Connecting to namenode via http://hadoop002:50070
FSCK started by hadoop (auth:SIMPLE) from /172.18.32.193 for path / at Sun Apr 07 21:51:36 CST 2019
..
/blockrecover/banking.txt:  Under replicated BP-1152973221-172.18.32.193-1554134553451:blk_1073741845_1021. Target Replicas is 3 but found 2 replica(s).
......Status: HEALTHY
 Total size:	484118690 B
 Total dirs:	21
 Total files:	8
 Total symlinks:		0
 Total blocks (validated):	9 (avg. block size 53790965 B)
 Minimally replicated blocks:	9 (100.0 %)
 Over-replicated blocks:	0 (0.0 %)
 Under-replicated blocks:	1 (11.111111 %)
 Mis-replicated blocks:		0 (0.0 %)
 Default replication factor:	3
 Average block replication:	2.8888888
 Corrupt blocks:		0
 Missing replicas:		1 (3.7037036 %)
 Number of data-nodes:		3
 Number of racks:		1
FSCK ended at Sun Apr 07 21:51:36 CST 2019 in 15 milliseconds


The filesystem under path '/' is HEALTHY

过了几分钟，数据修复了

[hadoop@hadoop001 subdir0]$ hdfs fsck /
19/04/07 21:55:25 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Connecting to namenode via http://hadoop002:50070
FSCK started by hadoop (auth:SIMPLE) from /172.18.32.193 for path / at Sun Apr 07 21:55:26 CST 2019
........Status: HEALTHY
 Total size:	484118690 B
 Total dirs:	21
 Total files:	8
 Total symlinks:		0
 Total blocks (validated):	9 (avg. block size 53790965 B)
 Minimally replicated blocks:	9 (100.0 %)
 Over-replicated blocks:	0 (0.0 %)
 Under-replicated blocks:	0 (0.0 %)
 Mis-replicated blocks:		0 (0.0 %)
 Default replication factor:	3
 Average block replication:	3.0
 Corrupt blocks:		0
 Missing replicas:		0 (0.0 %)
 Number of data-nodes:		3
 Number of racks:		1
FSCK ended at Sun Apr 07 21:55:26 CST 2019 in 5 milliseconds


The filesystem under path '/' is HEALTHY

再来一次，这次手动来修复

[hadoop@hadoop001 subdir0]$ ls
blk_1073741825            blk_1073741841            blk_1073741844
blk_1073741825_1001.meta  blk_1073741841_1017.meta  blk_1073741844_1020.meta
blk_1073741839            blk_1073741842            blk_1073741845
blk_1073741839_1015.meta  blk_1073741842_1018.meta  blk_1073741845_1021.meta
blk_1073741840            blk_1073741843            blk_1073741846
blk_1073741840_1016.meta  blk_1073741843_1019.meta  blk_1073741846_1022.meta

[hadoop@hadoop001 subdir0]$ rm -fr blk_1073741845 blk_1073741845_1021.meta

[hadoop@hadoop001 subdir0]$ ls
blk_1073741825            blk_1073741841            blk_1073741844
blk_1073741825_1001.meta  blk_1073741841_1017.meta  blk_1073741844_1020.meta
blk_1073741839            blk_1073741842            blk_1073741846
blk_1073741839_1015.meta  blk_1073741842_1018.meta  blk_1073741846_1022.meta
blk_1073741840            blk_1073741843
blk_1073741840_1016.meta  blk_1073741843_1019.meta
[hadoop@hadoop001 subdir0]$ stop-dfs.sh

[hadoop@hadoop001 subdir0]$ start-dfs.sh

重启以后，可以看到有一个副本丢失，而且ls可以看到删除的文件还没有被自动恢复 

[hadoop@hadoop001 subdir0]$ hdfs fsck /
19/04/07 22:07:45 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Connecting to namenode via http://hadoop002:50070
FSCK started by hadoop (auth:SIMPLE) from /172.18.32.193 for path / at Sun Apr 07 22:07:46 CST 2019
..
/blockrecover/banking.txt:  Under replicated BP-1152973221-172.18.32.193-1554134553451:blk_1073741845_1021. Target Replicas is 3 but found 2 replica(s).
......Status: HEALTHY
 Total size:	484118690 B
 Total dirs:	21
 Total files:	8
 Total symlinks:		0
 Total blocks (validated):	9 (avg. block size 53790965 B)
 Minimally replicated blocks:	9 (100.0 %)
 Over-replicated blocks:	0 (0.0 %)
 Under-replicated blocks:	1 (11.111111 %)
 Mis-replicated blocks:		0 (0.0 %)
 Default replication factor:	3
 Average block replication:	2.8888888
 Corrupt blocks:		0
 Missing replicas:		1 (3.7037036 %)
 Number of data-nodes:		3
 Number of racks:		1
FSCK ended at Sun Apr 07 22:07:46 CST 2019 in 108 milliseconds


The filesystem under path '/' is HEALTHY

 

[hadoop@hadoop001 subdir0]$ ls
blk_1073741825            blk_1073741841            blk_1073741844
blk_1073741825_1001.meta  blk_1073741841_1017.meta  blk_1073741844_1020.meta
blk_1073741839            blk_1073741842            blk_1073741846
blk_1073741839_1015.meta  blk_1073741842_1018.meta  blk_1073741846_1022.meta
blk_1073741840            blk_1073741843
blk_1073741840_1016.meta  blk_1073741843_1019.meta

手动修复 

[hadoop@hadoop001 subdir0]$ hdfs debug recoverLease -path /blockrecover/banking.txt -retries 10
19/04/07 22:07:58 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

recoverLease SUCCEEDED on /blockrecover/banking.txt

成功了 

[hadoop@hadoop001 subdir0]$ hdfs fsck /
19/04/07 22:08:16 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Connecting to namenode via http://hadoop002:50070
FSCK started by hadoop (auth:SIMPLE) from /172.18.32.193 for path / at Sun Apr 07 22:08:17 CST 2019
........Status: HEALTHY
 Total size:	484118690 B
 Total dirs:	21
 Total files:	8
 Total symlinks:		0
 Total blocks (validated):	9 (avg. block size 53790965 B)
 Minimally replicated blocks:	9 (100.0 %)
 Over-replicated blocks:	0 (0.0 %)
 Under-replicated blocks:	0 (0.0 %)
 Mis-replicated blocks:		0 (0.0 %)
 Default replication factor:	3
 Average block replication:	3.0
 Corrupt blocks:		0
 Missing replicas:		0 (0.0 %)
 Number of data-nodes:		3
 Number of racks:		1
FSCK ended at Sun Apr 07 22:08:17 CST 2019 in 5 milliseconds


The filesystem under path '/' is HEALTHY
[hadoop@hadoop001 subdir0]$ ls
blk_1073741825            blk_1073741841            blk_1073741844
blk_1073741825_1001.meta  blk_1073741841_1017.meta  blk_1073741844_1020.meta
blk_1073741839            blk_1073741842            blk_1073741845
blk_1073741839_1015.meta  blk_1073741842_1018.meta  blk_1073741845_1021.meta
blk_1073741840            blk_1073741843            blk_1073741846
blk_1073741840_1016.meta  blk_1073741843_1019.meta  blk_1073741846_1022.meta

 

AaronLwx
关注
专栏目录
HDFS故障排除-NameNode故障处理
weixin_44216665的博客
08-20 692
NameNode故障处理
2021SC@SDUSC hadoop源码分析（八）
qq_53102935的博客
11-15 965
2021SC@SDUSC 本篇我将对hadoop-tools中的Rumen进行分析 Rumen 概观 Rumen是一个数据提取和分析工具Apache Hadoop。Rumenmines工作历史日志提取有意义的数据，并以易于解析、压缩的格式或摘要。来自MapReduce日志的原始跟踪数据通常不足以进行模拟、仿真和基准测试，因为这些工具通常试图测量源数据中没有出现的情况。例如，如果一个任务在原始跟踪数据中本地运行，但调度程序的模拟选择在远程机架上运行该任务，模拟器需要一个其输入无法提供的运行时。为了..
hive报错Could not get block locations. Source file “/user/hive/warehouse/xxx
博客
12-24 3394
hive报错 Could not get block locations. Source file "/user/hive/warehouse/xxx... 以及 Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Error while processing row (tag=0)... 是因为链接超时了设置set mapred.task.timeout=600000;为10分钟或更长即可 ...
hadoop常见错误及解决办法！
热门推荐
沉底的石头
05-30 1万+
1：Shuffle Error: Exceeded MAX_FAILED_UNIQUE_FETCHES; bailing-out  Answer： 程序里面需要打开多个文件，进行分析，系统一般默认数量是1024，（用ulimit -a可以看到）对于正常使用是够了，但是对于程序来讲，就太少了。 修改办法： 修改2个文件。        /etc/security/limits.
Hadoop采坑小记
Source_00的博客
10-20 1966
一 eclipse 上传文件到hdfs报错： java.io.IOException: Could not get block locations. Source file "/1108/daxian/banzhang/kafka_2.12-2.0.0.tgz" - Aborting... at org.apache.hadoop.hdfs.DataStreamer.setupPipeline...
Hadoop的jobhistoryserver配置
过河卒子Rover的博客
01-19 1万+
JobHistoryServer配置
HDFS故障恢复流程：从灾难到全面恢复的步骤
![HDFS故障恢复流程：从灾难到全面...本章将对HDFS故障恢复的必要性和基本原理进行概述，为后续章节深入探讨故障分类、诊断和具体恢复操作提供背景知识。 HDFS作为一个高度容错的系统，在设计之初就考虑到了故障恢复
【HDFS故障恢复】：快速诊断与恢复的实用手册
![【HDFS故障恢复】：快速... HDFS故障恢复概述 在分布式存储系统中，数据安全和可靠性是至关重要的。Hadoop分布式文件系统（HDFS）作为大数据处理的核心存储组件，其稳定性和故障恢复能力对于整个数据处理流程至关重
Hadoop原理详解：HDFS故障恢复与MapReduce实战
Hadoop的核心组成部分包括分布式文件系统 Hadoop Distributed File System (HDFS) 和 MapReduce 用于处理大规模数据处理任务。 HDFS的设计原则基于硬件错误的常态，它采用简单的一次写多次读一致性模型，适用于...
HDFS故障恢复机制揭秘：NameNode、DataNode、Secondary NameNode的角色
Hadoop分布式文件系统（HDFS）简介 Hadoop分布式文件系统（Hadoop Distributed File System，HDFS）是Apache Hadoop框架的核心组件之一，用于存储和管理大规模数据集。在本章中，我们将介绍HDFS的基本概念以及其...
CDH 丢失块及副本不足的块
只是甲的博客
03-11 3509
备注: CDH 6.3.1 文章目录一.问题描述二.解决方案2.1 丢失块查找2.2 解决oozie副本块不足的问题2.2.1 设置3个副本模式2.2.2 删除坏的块:参考 一.问题描述 如下图所示，刚安装的CDH,提示存在丢失块，也存在副本不足的块 二.解决方案 2.1 丢失块查找 sudo -u hdfs hadoop fsck / -files -blocks 测试记录: FSCK started by hdfs (auth:SIMPLE) from /10.31.1.123 for path /
搭建hadoop集群报错
m0_51142173的博客
05-23 460
1、将hdfs和yarn的启动命令执行反了----------解决：jps查看启动错的命令，用kill杀死该错误进程（kill 进程号） 2、start启动时将stop也复制进入执行命令---------解决：直接删除logs内文件，重新格式化 3、在修改配置文件时不小心删除了标签中的一个，或者多复制了该标签-----------解决：sudo vi修改，重新格式化 --------查看错误原因： 前往/opt/hadoop/logs文件中查看hdfs或yarn对应的log文件，根据报错解决问题。 （注意：
Hadoop distcp命令遇到的异常及解决方案
qq_24095055的博客
01-18 1717
1. Caused by: java.io.IOException: Mismatch in length of source:hdfs://xxx and target:hdfs://xxx 异常信息 Caused by: java.io.IOException: Mismatch in length of source:hdfs://xxx and target:hdfs://xxx 原因 需要远程复制的文件没有关闭，还处于写的状态。 解决方案 1 . 检查文件状态 hdfs fsck hdfs://1
hadoop-深入理解MapReduce(一)-Job提交流程
Airy的博客
01-20 2001
1.Job提交 先图解大致流程，心里有大概框架。 首先第一步进入waitForCompletion函数中的submit函数 进入sumit函数，重点关注connect函数 初始化 总结来说，上面过程就是建立连接，创建提交job的代理，判断是local还是yarn客户端 然后我们回到submit函数，继续看connect下面的部分 进入submitJobInternal函数 分析checkSpecs函数 ，检查输出路径是否有问题。 其中checkOutP...
YARN 作业 Staging目录的用途及配置
MAYIBO—BG的专栏
12-17 5089
目录staging目录的用途MapReduce作业staging的配置示例Spark 作业Staging目录的配置示例 staging目录的用途 关于staging目录可能很多人都不太会关注，毕竟日常运行作业也用不到这些配置。不过了解它对于我们理解作业的执行流程也是有所帮助的，比如我们都会使用hadoop jar 或 spark-submit等命令来提交一个MR或Spark作业，然后我们就会看到在集群的某些计算节点上启动executor(MapRedece对应的是mapper和reducer)来执行任务。这
Hadoop MapReduce Job提交后的交互日志
BIT工匠
12-09 6290
通过混合 NamdeNode， DataNode， ResourceManager，NodeManager的日志输出， 和 提交MapReduce Job的console输出日志，形成时间序列上的日志输出。 这样可以方便查看从client端提交job，整个hadoop的内部交互的大致过程。 MapReduce Job的console输出来自执行简单编写的例子： bin/hadoop j
配置启动hadoop jobhistory
帆了个帆的专栏
08-18 1443
博客地址:http://www.fanlegefan.com 文章地址:http://www.fanlegefan.com/index.php/2017/08/18/hadoop-jobhistory/编辑mapred-site.xml<property> <name>mapreduce.jobhistory.address</name> <value>0.0.0.0:10020<
hadoop 依赖式job_Hadoop集群的JobHistoryServer详解(转载）
weixin_40008135的博客
12-20 386
Hadoop自带了一个历史服务器，可以通过历史服务器查看已经运行完的Mapreduce作业记录，比如用了多少个Map、用了多少个Reduce、作业提交时间、作业启动时间、作业完成时间等信息。默认情况下，Hadoop历史服务器是没有启动的，我们可以通过下面的命令来启动Hadoop历史服务器如何启动历史服务？# mr-jobhistory-daemon.sh start historyserver这样...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值