当遇到如下错误的时候 可能以为是regionserver 挂掉或者其他原因导致连接不上regionserver 但后面提示了Hbase 表statistic_login 具体信息
Thu Jan 17 15:30:12 CST 2013, org.apache.hadoop.hbase.client.ScannerCallable@5ec136e9, org.apache.hadoop.hbase.NotServingRegionException: org.apache.hadoop.hbase.NotServingRegionException: Region is not online: statistic_login,2013-01-13 14:57:34:868850013488456,1358244290816.94deb62960731a30e0a009b39350b2ed.at org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:183)at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:205)at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:120)at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:628)at org.apache.hadoop.hbase.mapreduce.TableRecordReaderImpl.restart(TableRecordReaderImpl.java:80)at org.apache.hadoop.hbase.mapreduce.TableRecordReaderImpl.initialize(TableRecordReaderImpl.java:142)at org.apache.hadoop.hbase.mapreduce.TableRecordReader.initialize(TableRecordReader.java:122)at org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.createRecordReader(TableInputFormatBase.java:131)at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.<init>(MapTask.java:489)at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:731)at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)at org.apache.hadoop.mapred.Child$4.run(Child.java:255)at java.security.AccessController.doPrivileged(Native Method)at javax.security.auth.Subject.doAs(Subject.java:415)at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)at org.apache.hadoop.mapred.Child.main(Child.java:249)
此时使用hadoop fsck 进行分析 就能看到CORRUPT 的storefile路径 hadoop fs -rm 当前storefile
所以分析原因是 数据损坏大致当前数据存放的regin无法使用,状态应该为CORRUPT 此时 需要手动清除损坏的块 后,才能继续mapreduce