终于配置好了完全分布式的HBase1.2.4的环境,zookeeper集群是独立的,即HBase没有托管ZooKeeper的集群,研究清楚了新版本的api的变化,遂写好了简单的demo代码往某张表里put数据,api很简单从略,运行时卡住,程序一直卡着不动,也没有结束,执行到put那一行就彻底卡住了,一个小时后,控制台会报错:
2017-03-29 16:01:48 ERROR org.apache.hadoop.hbase.client.AsyncProcess - Failed to get region location
java.io.IOException: Failed to get result within timeout, timeout=60000ms
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:206)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:60)
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:210)
at org.apache.hadoop.hbase.client.ClientSmallReversedScanner.loadCache(ClientSmallReversedScanner.java:212)
at org.apache.hadoop.hbase.client.ClientSmallReversedScanner.next(ClientSmallReversedScanner.java:186)
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(ConnectionManager.java:1275)
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1181)
at org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:410)
at org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:359)
at org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:238)
at org.apache.hadoop.hbase.client.BufferedMutatorImpl.flush(BufferedMutatorImpl.java:190)
at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1434)
at org.apache.hadoop.hbase.client.HTable.put(HTable.java:1018)
at test.com.rz.hbase.HBaseUtilTest.main(HBaseUtilTest.java:34)
[ERROR]:save net data error! An error occured when saving net data!the current rowKey is [[B@218a968b]
org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 1 action: IOException: 1 time,
at org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(AsyncProcess.java:258)
at org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$2000(AsyncProcess.java:238)
at org.apache.hadoop.hbase.client.AsyncProcess.waitForAllPreviousOpsAndReset(AsyncProcess.java:1817)
at org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:240)
at org.apache.hadoop.hbase.client.BufferedMutatorImpl.flush(BufferedMutatorImpl.java:190)
at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1434)
at org.apache.hadoop.hbase.client.HTable.put(HTable.java:1018)
at test.com.rz.hbase.HBaseUtilTest.main(HBaseUtilTest.java:34)
这个问题困扰了我很多天,我也从网上查了很多资料,综合分析,可能的原因如下:
1. hadoop区块损坏,用相关命令检查。
2,.zookeeper的meta数据损坏引起的问题。
3. 某个节点不正常时,可能也会引起此问题
4.hbase 1.0以前的版本里有一个bug会引起此问题
5.如果代码里写错了column family的名字,也可能会引起此问题
6. hbase 与hadoop版本不兼容也会引发此问题
7. 完全分布式的hbase,如果没有用hadoop集群的hadoop相关的jar替换掉 hbase里对应的jar也可能会引起莫名其妙的问题
8. 我还甚至怀疑是Hbase1.2.4的版本自身的问题
9. 服务器防火墙的问题
10.运行 demo程序的客户端防火墙的问题
然而,我逐一排查,我的问题不是以上原因引起的。
最后这个问题终于被我成功的解决了,那么,亲,你想知道怎么解决的吗?