【elasticsearch】问题解决：Remote responded with a chunk that was too large. Use a smaller batch size

一、前言

在使用reindex的方式来进行es索引跨集群迁移时，遇到的报错，这里整理并记录一下。

---

二、问题描述

es索引迁移命令：

curl -u admin:admin   -XPOST "http://10.1.36.14:9200/_reindex" -d '
{
  "source": {
    "remote": {
      "host": "http://10.11.36.142:29200",
      "username": "admin",
      "password": "admin"
    },
    "index": "sentiment_new_short_data_2020_06"
  },
  "dest": {
    "index": "sentiment_new_short_data_2020_06"
  }
}'

过了一段时间后报错，报错内容如下：

{
	"error": {
		"root_cause": [{
			"type": "illegal_argument_exception",
			"reason": "Remote responded with a chunk that was too large. Use a smaller batch size."
		}],
		"type": "illegal_argument_exception",
		"reason": "Remote responded with a chunk that was too large. Use a smaller batch size.",
		"caused_by": {
			"type": "content_too_long_exception",
			"reason": "entity content is too long [120409280] for the configured buffer limit [104857600]"
		}
	},
	"status": 400
}

三、解决方案：

方法一：

修改HTTP 请求正文的最大大小。默认为100M，(Static) Maximum size of an HTTP request body. Defaults to 100mb。

http.max_content_length

注意事项：静态设置只能在未启动或关闭的节点上使用 elasticsearch.yml。
必须在集群中的每个相关节点上设置静态设置。

方法二：

减少迁移时每批次的文档数量，默认为1000

{ "size": 800}

修改后：

curl -u admin:admin   -XPOST "http://10.1.36.14:9200/_reindex" -d '
{
  "source": {
    "remote": {
      "host": "http://10.11.36.142:29200",
      "username": "admin",
      "password": "admin"
    },
    "index": "sentiment_new_short_data_2020_06",
    "size": 800
  },
  "dest": {
    "index": "sentiment_new_short_data_2020_06"
  }
}'