Hbase异常宕机，NoServerForRegionException: Unable to find region（2）

最新推荐文章于 2022-03-29 12:41:13 发布

javenzhen

最新推荐文章于 2022-03-29 12:41:13 发布

阅读量1.3k

点赞数

分类专栏： BigData 文章标签： hbase 宕机异常故障

本文链接：https://blog.csdn.net/javenzhen/article/details/48412725

版权

BigData 专栏收录该内容

2 篇文章 0 订阅

订阅专栏

1、查看META元数据表的信息


    hbase(main):001:0> scan '.META.'
    ROW COLUMN+CELL
     hivetest,,1441805677008.42b9d2fff35c183 column=info:regioninfo, timestamp=1441805677121, value={NAME => 'hivetest,,1441805677008.42b9d2fff35c18393058668f77e
     93058668f77e7b86e. 7b86e.', STARTKEY => '', ENDKEY => '', ENCODED => 42b9d2fff35c18393058668f77e7b86e,}
     hivetest,,1441805677008.42b9d2fff35c183 column=info:server, timestamp=1441991117154, value=slave1:60020
     93058668f77e7b86e.
     hivetest,,1441805677008.42b9d2fff35c183 column=info:serverstartcode, timestamp=1441991117154, value=1441989279388
     93058668f77e7b86e.
     test,,1441718128275.edca99f7a3a627594f2 column=info:regioninfo, timestamp=1441718128579, value={NAME => 'test,,1441718128275.edca99f7a3a627594f2ab1af7e7cb1a
     ab1af7e7cb1ad. d.', STARTKEY => '', ENDKEY => '', ENCODED => edca99f7a3a627594f2ab1af7e7cb1ad,}
     test,,1441718128275.edca99f7a3a627594f2 column=info:server, timestamp=1441991117268, value=slave2:60020
     ab1af7e7cb1ad.
     test,,1441718128275.edca99f7a3a627594f2 column=info:serverstartcode, timestamp=1441991117268, value=1441989279626
     ab1af7e7cb1ad.
    2 row(s) in 1.3030 seconds

2、查看Hbase UI表的信息

在UI中看见之前创建的mytab表的10个region没有了，但是在hdfs中的文件任然存在
3、使用hbck的-fixAssignments尝试信息

    grid@master1:~$ hbase hbck -fixAssignments
    ..........................................................................
    Summary:
      -ROOT- is okay.
        Number of regions: 1
        Deployed on: slave2,60020,1442009098829
      .META. is okay.
        Number of regions: 1
        Deployed on: slave2,60020,1442009098829
      hivetest is okay.
        Number of regions: 1
        Deployed on: slave1,60020,1442009098138
      test is okay.
        Number of regions: 1
        Deployed on: slave2,60020,1442009098829
    11 inconsistencies detected.
    Status: INCONSISTENT
    15/09/11 22:17:39 INFO client.HConnectionManager$HConnectionImplementation: Closed zookeeper sessionid=0x464fbd40e2bc0008
    15/09/11 22:17:39 INFO zookeeper.ClientCnxn: EventThread shut down
    15/09/11 22:17:39 INFO zookeeper.ZooKeeper: Session: 0x464fbd40e2bc0008 closed

没有修复成功，检测到了11个不一致的信息。
4、再次使用 hbck的-fixHdfsOrphans尝试信息

    grid@master1:~$ hbase hbck -fixHdfsOrphans
    .............................................................
    Summary:
      -ROOT- is okay.
        Number of regions: 1
        Deployed on: slave2,60020,1442009098829
      .META. is okay.
        Number of regions: 1
        Deployed on: slave2,60020,1442009098829
      hivetest is okay.
        Number of regions: 1
        Deployed on: slave1,60020,1442009098138
      test is okay.
        Number of regions: 1
        Deployed on: slave2,60020,1442009098829
    11 inconsistencies detected.
    Status: INCONSISTENT
    15/09/11 22:13:50 INFO client.HConnectionManager$HConnectionImplementation: Closed zookeeper sessionid=0x464fbd40e2bc0007
    15/09/11 22:13:50 INFO zookeeper.ZooKeeper: Session: 0x464fbd40e2bc0007 closed
    15/09/11 22:13:50 INFO zookeeper.ClientCnxn: EventThread shut down

还是没有修复成功
5、使用 hbck的-repair修复

    grid@master1:~$ hbase hbck -repair
    ..............................................................
    Summary:
      -ROOT- is okay.
        Number of regions: 1
        Deployed on: slave2,60020,1442009098829
      .META. is okay.
        Number of regions: 1
        Deployed on: slave2,60020,1442009098829
      hivetest is okay.
        Number of regions: 1
        Deployed on: slave1,60020,1442009098138
      mytab is okay.
        Number of regions: 11
        Deployed on: slave1,60020,1442009098138 slave2,60020,1442009098829 slave3,60020,1442009098765
      test is okay.
        Number of regions: 1
        Deployed on: slave2,60020,1442009098829
    0 inconsistencies detected.
    Status: OK

此次修复成功，找回了mytab表。
6、再次去Hbase UI查看表的信息

太高兴了，表mytab找回来了。
7、介绍Hbase hbck的修复选项

    -fix，向下兼容用，被-fixAssignments替代
    -fixAssignments，用于修复region assignments错误
    -fixMeta，用于修复meta表的问题，前提是HDFS上面的region info信息有并且正确。
    -fixHdfsHoles，修复region holes（空洞，某个区间没有region）问题
    -fixHdfsOrphans，修复Orphan region（hdfs上面没有.regioninfo的region）
    -fixHdfsOverlaps，修复region overlaps（区间重叠）问题
    -fixVersionFile，修复缺失hbase.version文件的问题
    -maxMerge <n> （n默认是5），当region有重叠是，需要合并region，一次合并的region数最大不超过这个值。
    -sidelineBigOverlaps ，当修复region overlaps问题时，允许跟其他region重叠次数最多的一些region不参与（修复后，可以把没有参与的数据通过bulk load加    载到相应的region）
    -maxOverlapsToSideline <n> （n默认是2），当修复region overlaps问题时，一组里最多允许多少个region不参与由于选项较多，所以有两个简写的选项
    -repair，相当于-fixAssignments -fixMeta -fixHdfsHoles -fixHdfsOrphans -fixHdfsOverlaps -fixVersionFile -sidelineBigOverlaps
    -repairHoles，相当于-fixAssignments -fixMeta -fixHdfsHoles -fixHdfsOrphans