ElasticSearch提示java.lang.IllegalArgumentException: Document contains at least one immense term in .

已于 2023-06-01 21:28:03 修改
阅读量2.3k 收藏
点赞数
分类专栏： ElasticSearch 文章标签： elasticsearch java lucene
于 2021-07-03 15:06:42 首次发布
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/qq_38423256/article/details/118440093
版权

 翻译：索引文档中包含了一个巨大的字段



java.lang.IllegalArgumentException: Document contains at least one immense term in field="responseData" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped.  Please correct the analyzer to not produce such terms.  The prefix of the first immense term is: '[34, 91, 123, 92, 34, 99, 97, 112, 116, 105, 111, 110, 92, 34, 58, 92, 34, -26, -75, -117, -24, -81, -107, -26, -75, -117, -24, -81, -107, -29]...', original message: bytes can be at most 32766 in length; got 41756
	at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:772)
	at org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:417)
	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:373)
	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:231)
	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:478)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1562)
	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1307)
	at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:558)
	at org.elasticsearch.index.engine.InternalEngine.innerIndex(InternalEngine.java:520)
	at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:409)
	at org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:556)
	at org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:546)
	at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:191)
	at org.elasticsearch.action.index.TransportIndexAction.onPrimaryShard(TransportIndexAction.java:144)
	at org.elasticsearch.action.index.TransportIndexAction.onPrimaryShard(TransportIndexAction.java:63)
	at org.elasticsearch.action.support.replication.TransportWriteAction.shardOperationOnPrimary(TransportWriteAction.java:75)
	at org.elasticsearch.action.support.replication.TransportWriteAction.shardOperationOnPrimary(TransportWriteAction.java:48)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:905)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:875)
	at org.elasticsearch.action.support.replication.ReplicationOperation.execute(ReplicationOperation.java:113)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.onResponse(TransportReplicationAction.java:323)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.onResponse(TransportReplicationAction.java:258)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$1.onResponse(TransportReplicationAction.java:855)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$1.onResponse(TransportReplicationAction.java:852)
	at org.elasticsearch.index.shard.IndexShardOperationsLock.acquire(IndexShardOperationsLock.java:142)
	at org.elasticsearch.index.shard.IndexShard.acquirePrimaryOperationLock(IndexShard.java:1655)
	at org.elasticsearch.action.support.replication.TransportReplicationAction.acquirePrimaryShardReference(TransportReplicationAction.java:864)
	at org.elasticsearch.action.support.replication.TransportReplicationAction.access$400(TransportReplicationAction.java:90)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.doRun(TransportReplicationAction.java:275)
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:254)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:246)
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69)
	at org.elasticsearch.transport.TransportService$6.doRun(TransportService.java:577)
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:527)
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: NotSerializableExceptionWrapper[max_bytes_length_exceeded_exception: bytes can be at most 32766 in length; got 41756]
	at org.apache.lucene.util.BytesRefHash.add(BytesRefHash.java:263)
	at org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:149)
	at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:762)
	... 38 more
java.lang.IllegalArgumentException: Document contains at least one immense term in field="responseData" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped.  Please correct the analyzer to not produce such terms.  The prefix of the first immense term is: '[34, 123, 92, 34, 100, 97, 116, 97, 92, 34, 58, 91, 123, 92, 34, 105, 100, 92, 34, 58, 92, 34, 97, 57, 57, 51, 48, 48, 100, 56]...', original message: bytes can be at most 32766 in length; got 62581
	at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:772)
	at org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:417)
	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:373)
	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:231)
	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:478)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1562)
	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1307)
	at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:558)
	at org.elasticsearch.index.engine.InternalEngine.innerIndex(InternalEngine.java:520)
	at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:409)
	at org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:556)
	at org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:546)
	at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:191)
	at org.elasticsearch.action.index.TransportIndexAction.onPrimaryShard(TransportIndexAction.java:144)
	at org.elasticsearch.action.index.TransportIndexAction.onPrimaryShard(TransportIndexAction.java:63)
	at org.elasticsearch.action.support.replication.TransportWriteAction.shardOperationOnPrimary(TransportWriteAction.java:75)
	at org.elasticsearch.action.support.replication.TransportWriteAction.shardOperationOnPrimary(TransportWriteAction.java:48)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:905)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:875)
	at org.elasticsearch.action.support.replication.ReplicationOperation.execute(ReplicationOperation.java:113)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.onResponse(TransportReplicationAction.java:323)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.onResponse(TransportReplicationAction.java:258)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$1.onResponse(TransportReplicationAction.java:855)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$1.onResponse(TransportReplicationAction.java:852)
	at org.elasticsearch.index.shard.IndexShardOperationsLock.acquire(IndexShardOperationsLock.java:142)
	at org.elasticsearch.index.shard.IndexShard.acquirePrimaryOperationLock(IndexShard.java:1655)
	at org.elasticsearch.action.support.replication.TransportReplicationAction.acquirePrimaryShardReference(TransportReplicationAction.java:864)
	at org.elasticsearch.action.support.replication.TransportReplicationAction.access$400(TransportReplicationAction.java:90)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.doRun(TransportReplicationAction.java:275)
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:254)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:246)
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69)
	at org.elasticsearch.transport.TransportService$6.doRun(TransportService.java:577)
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:527)
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: NotSerializableExceptionWrapper[max_bytes_length_exceeded_exception: bytes can be at most 32766 in length; got 62581]
	at org.apache.lucene.util.BytesRefHash.add(BytesRefHash.java:263)
	at org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:149)
	at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:762)
	... 38 more

提示信息说 responseData 超出了最大长度（the max length 32766）

解决方法：设置字段长度 ignore_above :256 （长度超过256个字节部分忽视）

ElasticSearch已经提供了解决方案，就是ignore_above示例配置如下：

curl -XPUT 'http://localhost:9200/twitter' -d '
{
"mappings":{
      "tweet" : {
         "properties" : {
               "message" : {"type" : "string", "index":"not_analyzed","ignore_above":256 }
                         }
                            }
                    }
            }
}
其中tweet下的message字段不做分词等处理，直接将原始内容来做索引，当内容长度大于256字节时，只索引前面256个字符，后面的内容被丢弃。这样就不会出现前文所提的immense term的错误了。

一般ignore_above设置就是为not_analyzed字段存在的，不可滥用。

参考：https://blog.csdn.net/iteye_6322/article/details/82647099

喜羊羊love红太狼
关注
  • 0
    点赞
  • 踩
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
遇到问题--k8s--helm安装elasticsearch报错java.lang.IllegalArgumentException: unknown setting [node.remote_clu
直到世界的尽头
12-03 2369
情况 使用helm安装elasticsearch 使用配置和命令如下: master.yaml --- nodeGroup: "master" roles: master: "true" ingest: "false" data: "false" replicas: 3 image: 123.dkr.ecr.cn-northwest-1.amazonaws.com.cn/k8s-env/elasticsearch imageTag: 7.6.2 resources: requests
解决java.lang.IllegalArgumentException异常的正确方法（通俗易懂）
最新发布
weixin_44361445的博客
05-11 2901
本论文将深入探讨java.lang.IllegalArgumentException异常的产生原因、常见情景以及正确解决方法，并提供丰富的代码示例和实践经验，帮助读者更好地理解和处理这一异常。异常处理：在方法内部对可能引发IllegalArgumentException异常的代码块进行try-catch处理，以捕获并处理异常，避免异常的传播和程序的异常终止。非法的方法参数：当调用方法时传入了不合法的参数，例如传入了空值或负数，就可能导致IllegalArgumentException异常的抛出。
es使用过程中遇到Document contains at least one immense term 的异常
m0_37541394的博客
08-30 2187
这里写自定义目录标题欢迎使用Markdown编辑器新的改变功能快捷键合理的创建标题，有助于目录的生成如何改变文本的样式插入链接与图片如何插入一段漂亮的代码片生成一个适合你的列表创建一个表格设定内容居中、居左、居右SmartyPants创建一个自定义列表如何创建一个注脚注释也是必不可少的KaTeX数学公式新的甘特图功能，丰富你的文章UML 图表FLowchart流程图导出与导入导出导入 欢迎使用Markdown编辑器 你好！ 这是你第一次使用 Markdown编辑器 所展示的欢迎页。如果你想学习如何使用Mar
Document contains at least one immense term in field=“****“ (whose UTF8 encoding is longer than t
vincent_duan的专栏
07-14 2095
使用的es版本是6.2.3，在入数据的时候，有些数据没有入进去，报了这个错误： java.lang.IllegalArgumentException: Document contains at least one immense term in field="message" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped. Pleas e correct the analyzer t
Elasticsearch Exception [type=illegal_argument_exception, reason=Document contains at least one imme
qq_40788718的博客
03-05 1413
Elasticsearch Exception [type=illegal_argument_exception, reason=Document contains at least one immense term in field=“xxxx”(whose UTF8 encoding is longer than the max length 32766),all of which were skipped. Please correct the analyzer to not produce such
ElasticSearch异常：java.lang.IllegalArgumentException: script_lang not supported
李玉志的博客
02-27 3071
异常原因 原因就是因为官方给的例子参数位置书写错误，查看官方文档代码如下： UpdateByQueryRequestBuilder updateByQuery = UpdateByQueryAction.INSTANCE.newRequestBuilder(client); updateByQuery.source("source_index") .script(new Script(...
springboot使用elasticsearch repository方式 报java.lang.IllegalArgumentException异常
桃之夭夭_zll
05-26 2751
详细异常信息 Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'bookRepository': Invocation of init method failed; nested exception is java.lang.IllegalArgu...
【kettle集成cdh6.1】hadoop file output浏览目录报错：java.lang.NoClassDefFoundError: com/ctc/wstx/io/SystemId
01-07
【kettle集成cdh6.1】外部数据源读写hdfs若干错前言环境报错问题分析解决办法后记 前言 最近试着上手了一下kettle，搭建过程很简单，就是下载个包解压一下，但是在配置数据源的过程中着实踩了不少坑，这里记录一下。...
最新版linux elasticsearch-7.16.1-linux-x86_64.tar.gz
12-16
启动Elasticsearch可以使用`bin/elasticsearch`脚本，但为了防止权限问题，建议以守护进程（daemon）方式运行，并指定`ES_JAVA_OPTS`环境变量控制JVM内存： ```bash export ES_JAVA_OPTS="-Xms2g -Xmx2g" # 设置JVM...
最新版windows elasticsearch-7.17.6-windows-x86_64.zip
09-02
Elasticsearch是一个开源的全文搜索引擎，它以其高效、可扩展和实时的数据检索能力而闻名。在Windows平台上，Elasticsearch的安装和配置过程与在其他操作系统上略有不同。本篇将详细阐述关于"最新版windows elastic...
elasticsearch-6.8.18.rar(elasticsearch-6.8.18.zip)
08-21
Elasticsearch是一个强大的开源搜索引擎，基于Apache Lucene构建，它为开发者和数据分析师提供了高效、可扩展、实时的搜索和分析功能。6.8.18版本是Elasticsearch的一个稳定版本，它在前几个版本的基础上进行了优化...
最新版linux elasticsearch-7.13.4-linux-x86_64.tar.gz
07-21
7. 由于Elasticsearch需要运行在Java环境下，确保已安装Java 8或更高版本，并设置好`JAVA_HOME`环境变量。 配置Elasticsearch： 1. 打开配置文件`/usr/local/elasticsearch/elasticsearch-7.13.4/config/elastic...
elasticsearch报错java.lang.IllegalStateException: failed to obtain node locks, tried [[/usr/local/elas
dsqcsdn的博客
09-09 9574
报错原因：没有正常停止elasticsearch 解决 #查看elastic的进程号 并杀死 ps aux | grep elasticsearch kill -9 进程号 #重启 -d 后台运行 ./bin/elasticsearch -d
Elasticsearch8.6启动异常：fatal exception while booting Elasticsearch
yang516521的博客
04-11 2617
elasticsearch8.6/8.7启动异常
Elasticsearch——》解决启动报错fatal exception while booting Elasticsearchjava.nio.file.InvalidPathException
小仙~
04-28 6013
2、现象 3、原因 环境变量classpath最后面空格！！！ 删除环境变量classpath最后面空格！！！再次重新启动！
Elasticsearch启动报错：fatal exception while booting Elasticsearchjava.lang.RuntimeException: can not run
不二青春
05-09 3774
Linux环境下Elasticsearch启动报错
ElasticSearch常见问题1：java.lang.IllegalArgumentException:Limit of total fields [1000] has been exceeded
weixin_38044597的博客
03-10 3144
1.出现问题 从MongoDB中向ElasticSearch中导入数据时，导入数据界面和ElasticSearch控制界面分别出现以下异常： 导入程序出现以下异常： ERROR 2021/03/10 17:06:00 index: [wdlacs__devicetest _doc 60488289290c6fd5947a6028 {illegal_argument_exception Limit of total fields [1000] has been exceeded %!s(boo
java.lang.IllegalArgumentExceptio:ik的报错
qq_34281601的博客
08-22 3631
java.lang.IllegalArgumentExceptio:Unknown Analyzer type [org.elasticsearch.index.analysis.IkAnalyzerProvider] for[ik] 错误提示的很明显   未知的分词器for ik 分析认为可能是 elasticsearch-2.4.0\config\elasticsearch.yml 配置错
有关elasticsearch报java.lang.IllegalStateException: Failed to load ApplicationContext的解决方法
weixin_45854212的博客
05-24 971
不得不说今天真的是蠢到家了，看到这个bug(注入bean失败异常), 我首先先caused by后的内容找到问题然后进行了以下通常报这个bug的几种情况下手： 配置文件是否出错 是否有字段写错了,没有对应上 mybatis-spring版本不支持当前mybatis版本和Spring版本（版本兼容性问题可以更改版本解决此类问题） 查看mysql数据库服务器是否开启 或数据库信息名称是否匹配 jdk版本过高导致的无法匹配(例如用jdk1.9以上的换取到jdk1.8能解决这个bug) 如果是jdk版本号过高的原
es StartupException: java.lang.IllegalArgumentException
01-31
es StartupException: java.lang.IllegalArgumentException是Elasticsearch启动过程中遇到的异常错误。这个错误通常是由于插件版本不匹配导致的。 根据提供的引用内容，可以看到错误信息中提到了一个插件名为...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值