Zookeeper 服务器端和客户端扩大节点数据1M大小限制

最新推荐文章于 2024-09-05 03:06:06 发布

Rainy酱

最新推荐文章于 2024-09-05 03:06:06 发布

阅读量1.6w

点赞数 6

分类专栏： ZooKeeper 文章标签： zookeeper 1m 节点数据 ConnectionLossException Unreasonable length

本文链接：https://blog.csdn.net/zyq_2014/article/details/85948902

版权

Zookeeper默认节点数据上限为1M，可通过修改jute.maxbuffer参数来扩大存储限制。该参数需在服务器端和客户端同时设置。当超过1M时，服务器端写入会抛出`ConnectionLossException`，客户端读取会抛出`IOException: Unreasonable length`。解决方案是分别在`zkServer.sh`和`zkCli.sh`中增加配置，并确保在代码中正确设置jute.maxbuffer属性。

摘要由CSDN通过智能技术生成

Zookeeper中每个结点默认的数据量上限是1M，如果需要存入大于1M的数据量，则要修改jute.maxbuffer参数，先来看下zookeeper中这个参数的意义

jute.maxbuffer： 默认值1048575，单位字节，用于配置单个数据节点（ZNode）上可以存储的最大数据大小。需要注意的是，在修改该参数的时候，需要在zookeeper集群的所有服务端以及客户端上设置才能生效。

接下来，我们就来看看，怎么设置这个参数吧。

服务器端：
如果服务器端没有修改该参数，直接写入过大数据会抛错：org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /test
在这里插入图片描述

解决办法：

zkServer.sh 新增-Djute.maxbuffer配置，这边以10M为例，具体大小需按实际情况修改（ZOO_USER_CFG为修改过部分的关键词）：

 #!/usr/bin/env bash
 
 # Licensed to the Apache Software Foundation (ASF) under one or more
 # contributor license agreements.  See the NOTICE file distributed with
 # this work for additional information regarding copyright ownership.
 # The ASF licenses this file to You under the Apache License, Version 2.0
 # (the "License"); you may not use this file except in compliance with
 # the License.  You may obtain a copy of the License at
 #
 #     http://www.apache.org/licenses/LICENSE-2.0
 #
 # Unless required by applicable law or agreed to in writing, software
 # distributed under the License is distributed on an "AS IS" BASIS,
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
 #
 # If this scripted is run out of /usr/bin or some other system bin directory
 # it should be linked to and not copied. Things like java jar files are found
 # relative to the canonical path of this script.
 #
 
 
 
 # use POSTIX interface, symlink is followed automatically
 ZOOBIN="${BASH_SOURCE-$0}"
 ZOOBIN="$(dirname "${ZOOBIN}")"
 ZOOBINDIR="$(cd "${ZOOBIN}"; pwd)"
 
 if [ -e "$ZOOBIN/../libexec/zkEnv.sh" ]; then
   . "$ZOOBINDIR/../libexec/zkEnv.sh"
 else
   . "$ZOOBINDIR/zkEnv.sh"
 fi
 
 # See the following page for extensive details on setting
 # up the JVM to accept JMX remote management:
 # http://java.sun.com/javase/6/docs/technotes/guides/management/agent.html
 # by default we allow local JMX connections
 
 ZO