Flink写入Elasticsearch（es）问题汇总记录

最新推荐文章于 2024-05-05 22:17:24 发布
最新推荐文章于 2024-05-05 22:17:24 发布
阅读量3.5k 收藏 2
点赞数
分类专栏： 大数据 文章标签： flink java es
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/qq_38558851/article/details/109081889
版权

Flink写入Elasticsearch（es）问题汇总

1、报错信息:

org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: mapping source must be pairs of fieldnames and properties definition.
	at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:302)
	at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:198)
	at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:149)
	at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:699)
	at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:232)
	at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:916)
	at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:992)
	at org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
	at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:992)
Caused by: java.lang.IllegalArgumentException: mapping source must be pairs of fieldnames and properties definition.
	at org.elasticsearch.action.admin.indices.mapping.put.PutMappingRequest.buildFromSimplifiedDef(PutMappingRequest.java:199)
	at org.elasticsearch.action.admin.indices.create.CreateIndexRequest.mapping(CreateIndexRequest.java:284)
	at org.zdww.flink.utils.EsUtils.mappings(EsUtils.java:137)
	at org.zdww.flink.utils.EsUtils.createIndex(EsUtils.java:75)
	at org.zdww.flink.sink.AddSinkElasticsearch.createIndex(AddSinkElasticsearch.java:104)
	at org.zdww.flink.sink.AddSinkElasticsearch.add(AddSinkElasticsearch.java:53)
	at org.zdww.flink.sink.AddSinkData.getExecConfig(AddSinkData.java:158)
	at org.zdww.flink.sink.AddSinkData.getModeConfig(AddSinkData.java:98)
	at org.zdww.flink.sink.AddSinkData.getExecSqlConfig(AddSinkData.java:70)
	at org.zdww.flink.sink.AddSinkData.add(AddSinkData.java:47)
	at org.zdww.flink.utils.ExecUtil.builder(ExecUtil.java:17)
	at org.zdww.flink.exec.MjzblJzlgblExec.exec(MjzblJzlgblExec.java:49)
	at org.zdww.flink.main.App.main(App.java:39)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:288)
	... 8 more

原因：maven导入的elasticsearch依赖版本过低，导入的包为过时的方法。
解决： mapping ()方法传入参数type，不修复导包，则mapping 方法需要加type参数_doc。或者自定义的tpye。
1)、从elasticsearch官网找到的

request.mapping("_doc",
“{\n” +
" “_doc”: {\n" +
" “properties”: {\n" +
" “message”: {\n" +
" “type”: “text”\n" +
" }\n" +
" }\n" +
" }\n" +
“}”,
XContentType.JSON);

改为

request.mapping("_doc", “datetime”, “type=date”);

多个参数

request.mapping("_doc", “datetime”, “type=date”, “string”, “type=text”);

注：es7后不用传入type参数，request.mapping( “datetime”, “type=date”);即可。
2、
报错信息

Exception in thread "main" ElasticsearchStatusException[Elasticsearch exception [type=illegal_argument_exception, reason=Unknown char_filter type [stconvert] for [tsconvert]]]
	at org.elasticsearch.rest.BytesRestResponse.errorFromXContent(BytesRestResponse.java:177)
	at org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:1406)
	at org.elasticsearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:1382)
	at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1269)
	at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1231)
	at org.elasticsearch.client.IndicesClient.create(IndicesClient.java:151)
	at org.zdww.flink.utils.EsUtils.createIndex(EsUtils.java:77)
	at org.zdww.flink.sink.AddSinkElasticsearch.createIndex(AddSinkElasticsearch.java:104)
	at org.zdww.flink.sink.AddSinkElasticsearch.add(AddSinkElasticsearch.java:53)
	at org.zdww.flink.sink.AddSinkData.getExecConfig(AddSinkData.java:158)
	at org.zdww.flink.sink.AddSinkData.getModeConfig(AddSinkData.java:98)
	at org.zdww.flink.sink.AddSinkData.getExecSqlConfig(AddSinkData.java:70)
	at org.zdww.flink.sink.AddSinkData.add(AddSinkData.java:47)
	at org.zdww.flink.utils.ExecUtil.builder(ExecUtil.java:17)
	at org.zdww.flink.exec.MjzblJzlgblExec.exec(MjzblJzlgblExec.java:49)
	at org.zdww.flink.main.App.main(App.java:39)
	Suppressed: org.elasticsearch.client.ResponseException: method [PUT], host [http://127.0.0.1:9200], URI [/gsww_mjzbl_jzlgbl_gss_test_flink?master_timeout=30s&timeout=30s], status line [HTTP/1.1 400 Bad Request]
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Unknown char_filter type [stconvert] for [tsconvert]"}],"type":"illegal_argument_exception","reason":"Unknown char_filter type [stconvert] for [tsconvert]"},"status":400}
		at org.elasticsearch.client.RestClient$SyncResponseListener.get(RestClient.java:920)
		at org.elasticsearch.client.RestClient.performRequest(RestClient.java:227)
		at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1256)
		... 12 more
	Caused by: org.elasticsearch.client.ResponseException: method [PUT], host [http://127.0.0.1:9200], URI [/gsww_mjzbl_jzlgbl_gss_test_flink?master_timeout=30s&timeout=30s], status line [HTTP/1.1 400 Bad Request]
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Unknown char_filter type [stconvert] for [tsconvert]"}],"type":"illegal_argument_exception","reason":"Unknown char_filter type [stconvert] for [tsconvert]"},"status":400}
		at org.elasticsearch.client.RestClient$1.completed(RestClient.java:540)
		at org.elasticsearch.client.RestClient$1.completed(RestClient.java:529)
		at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:119)
		at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:177)
		at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:436)
		at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:326)
		at org.apache.http.impl.nio.client.InternalRequestExecutor.inputReady(InternalRequestExecutor.java:83)
		at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265)
		at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81)
		at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39)
		at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114)
		at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
		at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
		at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
		at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
		at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
		at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588)
		at java.base/java.lang.Thread.run(Thread.java:834)

原因：elasticsearch没有安装插件stconvert

解决：安装对应插件

elasticsearch-6.4.2\plugins目录下安装插件，如下图：
在这里插入图片描述
3、报错
flink向es写入数据（es6）
报错信息如下：

2020-10-14 02:22:31,514 ERROR org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase [] - Failed Elasticsearch bulk request: Elasticsearch exception [type=illegal_argument_exception, reason=no write index is defined for alias [gsww_mjzbl_jzlgbl]. The write index may be explicitly disabled using is_write_index=false or the alias points to multiple indices without one being designated as a write index]
org.elasticsearch.ElasticsearchStatusException: Elasticsearch exception [type=illegal_argument_exception, reason=no write index is defined for alias [gsww_mjzbl_jzlgbl]. The write index may be explicitly disabled using is_write_index=false or the alias points to multiple indices without one being designated as a write index]
	at org.elasticsearch.rest.BytesRestResponse.errorFromXContent(BytesRestResponse.java:177) ~[blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:1406) ~[blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.elasticsearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:1382) ~[blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.elasticsearch.client.RestHighLevelClient$1.onFailure(RestHighLevelClient.java:1357) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.elasticsearch.client.RestClient$FailureTrackingResponseListener.onDefinitiveFailure(RestClient.java:844) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.elasticsearch.client.RestClient$1.completed(RestClient.java:548) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.elasticsearch.client.RestClient$1.completed(RestClient.java:529) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:119) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:177) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:436) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:326) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
	Suppressed: org.elasticsearch.client.ResponseException: method [POST], host [http://192.168.229.64:9200], URI [/_bulk?timeout=1m], status line [HTTP/1.1 400 Bad Request]
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"no write index is defined for alias [gsww_mjzbl_jzlgbl]. The write index may be explicitly disabled using is_write_index=false or the alias points to multiple indices without one being designated as a write index"}],"type":"illegal_argument_exception","reason":"no write index is defined for alias [gsww_mjzbl_jzlgbl]. The write index may be explicitly disabled using is_write_index=false or the alias points to multiple indices without one being designated as a write index"},"status":400}
		at org.elasticsearch.client.RestClient$1.completed(RestClient.java:540) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
		at org.elasticsearch.client.RestClient$1.completed(RestClient.java:529) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
		at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:119) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
		at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:177) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
		at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:436) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
		at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:326) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
		at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
		at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
		at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
		at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
		at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
		at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
		at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
		at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
		at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
		at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
		at java.lang.Thread.run(Thread.java:834) [?:?]
2020-10-14 02:22:32,173 ERROR org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase [] - Failed Elasticsearch bulk request: Elasticsearch exception [type=illegal_argument_exception, reason=no write index is defined for alias [gsww_mjzbl_jzlgbl]. The write index may be explicitly disabled using is_write_index=false or the alias points to multiple indices without one being designated as a write index]
org.elasticsearch.ElasticsearchStatusException: Elasticsearch exception [type=illegal_argument_exception, reason=no write index is defined for alias [gsww_mjzbl_jzlgbl]. The write index may be explicitly disabled using is_write_index=false or the alias points to multiple indices without one being designated as a write index]
	at org.elasticsearch.rest.BytesRestResponse.errorFromXContent(BytesRestResponse.java:177) ~[blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:1406) ~[blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.elasticsearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:1382) ~[blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.elasticsearch.client.RestHighLevelClient$1.onFailure(RestHighLevelClient.java:1357) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.elasticsearch.client.RestClient$FailureTrackingResponseListener.onDefinitiveFailure(RestClient.java:844) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.elasticsearch.client.RestClient$1.completed(RestClient.java:548) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.elasticsearch.client.RestClient$1.completed(RestClient.java:529) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:119) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:177) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:436) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:326) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
	Suppressed: org.elasticsearch.client.ResponseException: method [POST], host [http://192.168.229.64:9200], URI [/_bulk?timeout=1m], status line [HTTP/1.1 400 Bad Request]
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"no write index is defined for alias [gsww_mjzbl_jzlgbl]. The write index may be explicitly disabled using is_write_index=false or the alias points to multiple indices without one being designated as a write index"}],"type":"illegal_argument_exception","reason":"no write index is defined for alias [gsww_mjzbl_jzlgbl]. The write index may be explicitly disabled using is_write_index=false or the alias points to multiple indices without one being designated as a write index"},"status":400}
		at org.elasticsearch.client.RestClient$1.completed(RestClient.java:540) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
		at org.elasticsearch.client.RestClient$1.completed(RestClient.java:529) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
		at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:119) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
		at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:177) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
		at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:436) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
		at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:326) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
		at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
		at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
		at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
		at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
		at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
		at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
		at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
		at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
		at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
		at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588) [blob_p-ea8bfdc06b67c0436cb2f429dc7c27008aa5a6c9-04bf713032a7a4e7a3f9e6d3fdaa5f13:?]
		at java.lang.Thread.run(Thread.java:834) [?:?]

原因：别名指向多个索引
指向多个索引的别名，用于执行查询不需要做额外的设置，如果需要用于执行索引操作，则需要在被指向的多个索引中指定用于执行索引操作的索引，否则报错误。
解决：我的解决办法是修改别名，每个别名指向唯一的索引，可以根据实际情况修改。

qq_38558851
关注
  • 0
    点赞
  • 踩
  • 2
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
【异常】Flink整合ES出错，The implementation of the provided ElasticsearchSinkFunction is not serializable.
本本本添哥
03-21 589
flink输出（sink）到elasticsearch，ElasticsearchSinkFunction无法序列化问题。
flink任务调用ES报Request cannot be executed； I/O reactor status: STOPPED
dreamBoat的博客
04-27 1419
flink任务调用ES报Request cannot be executed; I/O reactor status: STOPPED
FlinkSql使用ES sink并指定主键，为什么数据还是会被覆盖？
最新发布
初阳
05-05 984
源码基于flink 1.14.4根据官方文档中对自定义sink的描述，connector sink的的工作原理如下元数据的内容由create table语句所定义，通过的实例表示，该实例表示Catalog中未解析的元数据，包含可以在create table语句中表示的所有特征，框架将其解析为实例（一个经过验证的CatalogTable），然后再将其传递给，最终传入到，用于insert into语句的写入。负责提供connector的具体实现逻辑。用于将的元数据转化为。其接口中仅有一个方法，返回实例。
从Flink往ElasticSearch进行Sink操作，报错circuit_breaking_exception
Jomly Kellenda的博客
05-11 1229
先贴报错： 2022-05-10 14:44:43 org.apache.flink.runtime.JobException: Recovery is suppressed by FixedDelayRestartBackoffTimeStrategy(maxNumberRestartAttempts=20, backoffTimeMS=10000) at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureH
Flink常见异常和错误信息小结
LittleMagic's Blog
03-22 7794
周末了，不想搞长篇大论，就写写这样的流水账吧。 Flink的常见异常众多，不可能面面俱到，所以想到哪儿写到哪儿，有漏掉的之后再补充。 部署和资源问题 (0) JDK版本过低 这不是个显式错误，但是JDK版本过低很有可能会导致Flink作业出现各种莫名其妙的问题，因此在生产环境中建议采用JDK 8的较高update（我们使用的是181）。 (1) Could not b...
Flink DataStream写入Elasticsearch与容错
王佩的CSDN博客
08-11 7432
本文基于Flink 1.8 和ES 6.2.4 总结Flink写入Elasticsearch的问题点。 在使用ElasticsearchSink时，需要注意以下几点: ElasticsearchSink内部使用RestHighLevelClient(ES >=6.x)或TransportClient(ES <6.x)与Elasticsearch集群进行通信。 Elasticse...
Flink执行常见报错总结
weixin_44464726的博客
04-17 6370
1、kafa 分区数小于应用部署节点数 处理方式：kafka 为topic增加partition;至少大于等于应用部署节点数 命令： bin/kafka-topics.sh --alter --zookeeper 10.112.179.12:2181 --partitions 10 --topic gome 2020-03-10 17:16:39,706 INFO org.apache.kafk...
flink写入es带有用户名密码
12-23
Flink以Elasticsearch7为sink 如何写入数据 。es有用户名密码。以代码的形式指导如何操作。并且具体的操作在压缩包中有说明文档。请仔细阅读
flink如何写入Elasticsearch5和Elasticsearch7
01-30
"doc") .source(json); } @Override public void process(Row element, RuntimeContext ...理解这些知识点后，开发者可以灵活地在Flink作业中将数据流写入Elasticsearch的不同版本，以满足日志存储、实时分析等需求。
flink1.11写入ES7.10完整代码（JAVA程序实现）
04-11
flink1.11写入ES7.10完整代码（JAVA程序实现）
flink采集文本数据写入ES，flink消费kafka数据写入ES,以及一些flink相关的demo
06-26
代码主要包含：flink采集文本数据写入ES，flink消费kafka数据写入ES,以及一些flink相关数据流相关的数据采集demo，附带有demo的技术文档，文档主要介绍了如何编译jar包，在flink的管理页面如何提交任务。 1、技术...
flink写入带kerberos认证的kudu connector
03-24
flink写入带kerberos认证的kudu connector
【Flink实战系列】Flink 各种报错汇总以及解决方案(持续更新中)
JasonLee实时计算
02-06 4453
这篇文章主要用来记录平时使用 Flink 过程中遇到的问题以及解决方案,其实前面已经有一篇这样的文章了,但是那篇文章里面的问题已经很多了,不太方便浏览,所以新开了一篇文章,后面的问题都会更新到这篇文章中. ...
记录一次Flink写入ES6的踩坑经历
weixin_40898246的博客
06-29 995
近日在一次Flink打包运行插入ES的任务时，遇到一些版本冲突的问题 Flink自己实现的连接器里提供的ES版本为6.3.1（6.x连接器版本）而我们集群环境则为6.5.4 运行时会报一些异常 java.lang.NoClassDefFoundError: org/apache/flink/streaming/connectors/elasticsearch/ActionRequestFailureHandler at com.guttv.test.AgDayHyperLogLogUv.main(AgDa
pyflink读文件写入到elasticsearch
zhaoyangjian724的专栏
06-04 651
创建 StreamExecutionEnvironment 对象。# 读取文件，创建 DataStream 对象。# 对每行数据添加字符串 'aaaa'
ES排序报错:Elasticsearch exception [type=illegal_argument_exception, reason=Text
weixin_62812758的博客
10-19 7289
对于需要每个文档字段数据(如聚合和排序)的操作，文本字段没有进行优化，因此这些操作在默认情况下是禁用的。请使用关键字字段代替。或者，在[createTime]上设置fielddata=true，以便通过反求倒排索引来加载字段数据。注意，这可能会使用有效内存。
Elasticsearch：使用 Apache Flink、Elasticsearch 打造实时事件处理及搜索
Elastic 中国社区官方博客
10-12 2543
从实时持续生成的数据中获取可操作的见解是当今许多企业的共同要求。 实时数据处理的一个广泛用例是仪表板。 支持此类用例的典型架构基于数据流处理器、具有低延迟读/写访问的数据存储和可视化框架。 在这篇博文中，我们演示了如何使用 Apache Flink 和 Elasticsearch 为流数据分析构建实时事件处理及搜索。 下图描述了我们的系统架构。在实际的应用中，我们可以使用 Kibana 共同打造实时仪表板解决方案。 在我们的架构中，Apache Flink 执行流分析作业，这些作业摄取数据流，应用转
FlinkCDC从mysql -kafka-es同步数据时遇到的问题
sghuu的博客
01-12 1615
2021-01-12 01:12:15,551 INFO org.apache.kafka.clients.consumer.internals.SubscriptionState [] - [Consumer clientId=consumer-10, groupId=null] Resetting offset for partition mysql2es_t_bank_question-1 to offset 0. 2021-01-12 01:12:15,924 ERROR org.apache.f
Flink启动yarn-session报错：Error while running the Flink session. java.lang.NoSuchMethodError
zhengzaifeidelushang的博客
05-19 1780
2021-05-19 13:18:35,976 ERROR org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Error while running the Flink session. java.lang.NoSuchMethodError: org.apache.commons.cli.Option.builder(Ljava/lang/String;)Lorg/apache/commons/cli/Option$Buil
java中flink写入es
08-11
在Java中使用Flink写入Elasticsearch(Es)可以通过以下几个步骤实现： 1. 配置Elasticsearch的连接信息：首先，需要在Java应用程序中配置Elasticsearch的连接信息，包括Elasticsearch的主机名、端口号和集群名称。 2. 创建Elasticsearch连接：Flink提供了ElasticsearchSink类，用于与Elasticsearch建立连接。通过创建ElasticsearchSink类的实例，并使用之前配置的连接信息初始化它，即可建立与Elasticsearch的连接。 3. 定义Elasticsearch的索引和类型：在将数据写入Elasticsearch之前，需要定义要写入的索引名称和类型名称。可以通过调用ElasticsearchSink类的withBulkFlushMaxActions()方法指定每个批次写入的最大记录数。 4. 将数据写入Elasticsearch：在Flink应用程序的数据流中，可以使用addSink()方法将ElasticsearchSink实例添加为数据流的下游。然后，可以使用ElasticsearchSink类的invoke()方法将数据写入Elasticsearch。 5. 启动Flink应用程序：在配置好Elasticsearch连接和数据写入逻辑后，可以使用Flink的ExecutionEnvironment或StreamExecutionEnvironment启动Flink应用程序。 6. 监控写入情况：在Flink应用程序运行期间，可以通过ElasticsearchSink类提供的方法来监控数据写入情况，包括写入成功和失败的记录数。 以上就是在Java中使用Flink写入Elasticsearch的步骤。通过配置Elasticsearch连接信息、创建连接、定义索引和类型、添加ElasticsearchSink实例以及监控写入情况，可以实现将Flink流处理的数据写入Elasticsearch的功能。

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值