上载数据时出错: KeyValue size too large

最新推荐文章于 2022-10-11 17:49:01 发布

hsg77

最新推荐文章于 2022-10-11 17:49:01 发布

阅读量1k

点赞数

分类专栏： geowave hbase

本文链接：https://blog.csdn.net/hsg77/article/details/88314905

版权

hbase 同时被 2 个专栏收录

10 篇文章 0 订阅

订阅专栏

geowave

7 篇文章 1 订阅

订阅专栏

上载数据时出错: KeyValue size too large
问题如下：

Exception in thread "main" java.lang.IllegalArgumentException: KeyValue size too large at 
org.apache.hadoop.hbase.client.HTable.validatePut(HTable.java:1545) at 
org.apache.hadoop.hbase.client.BufferedMutatorImpl.validatePut(BufferedMutatorImpl.java:175) at 
org.apache.hadoop.hbase.client.BufferedMutatorImpl.mutate(BufferedMutatorImpl.java:146) at 
mil.nga.giat.geowave.datastore.hbase.operations.HBaseWriter.writeMutations(HBaseWriter.java:94) at 
mil.nga.giat.geowave.datastore.hbase.operations.HBaseWriter.write(HBaseWriter.java:88) at 
mil.nga.giat.geowave.datastore.hbase.operations.HBaseWriter.write(HBaseWriter.java:81) at 
mil.nga.giat.geowave.core.store.base.BaseIndexWriter.write(BaseIndexWriter.java:99) at 
mil.nga.giat.geowave.core.store.base.BaseIndexWriter.write(BaseIndexWriter.java:72) at 
mil.nga.giat.geowave.core.store.index.writer.IndexCompositeWriter.write(IndexCompositeWriter.java:49) at 
com.cwgis.importFeatureHBase.importData(importFeatureHBase.java:120) at com.cwgis.App.import_db(App.java:115) at 
com.cwgis.App.main(App.java:51)

hbase. client. keyvalue. maxsize的设置问题

我的解决办法：
集群参数hbase-site.xml文件中的设置如下：

<!--keyval设置最大值500M-->
<property>
   <name>hbase.client.keyvalue.maxsize</name>
   <value>524288000</value>
</property>
<property>
  <name>hbase.server.keyvalue.maxsize</name>
  <value>524288000</value>    
</property>

重启hbase集群
客户端的lib库中修改hbase-shaded-client-1.4.6.jar和hbase-shaded-server-1.4.6.jar文件中hbase-default.xml参数

<property>
   <name>hbase.client.keyvalue.maxsize</name>
   <value>524288000</value>
</property>
<property>
  <name>hbase.server.keyvalue.maxsize</name>
  <value>524288000</value>    
</property>

—the—end----