Flink消费Kafka数据写入Elasticsearch

最新推荐文章于 2024-07-27 15:05:24 发布

帅气迷人的老害虫

最新推荐文章于 2024-07-27 15:05:24 发布

阅读量893

点赞数

文章标签： flink kafka elasticsearch 大数据

本文链接：https://blog.csdn.net/weixin_43861104/article/details/107409124

版权

本案例通过flink消费kafka数据，写入elasticsearch，自动生成索引，其中flink版本为1.10.0，es版本为6.4.3。

package com.bigdata.flink.sinks;

import com.bigdata.project.utils.FlinkUtils;
import org.apache.flink.api.common.functions.RuntimeContext;
import org.apache.flink.api.common.serialization.SimpleStringSchema;
import org.apache.flink.api.java.utils.ParameterTool;
import org.apache.flink.streaming.api.datastream.DataStream;
import org.apache.flink.streaming.api.datastream.DataStreamSource;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkFunction;
import org.apache.flink.streaming.connectors.elasticsearch.RequestIndexer;
import org.apache.flink.streaming.connectors.elasticsearch6.ElasticsearchSink;
import org.apache.flink.streaming.connectors.elasticsearch6.RestClientFactory;
import org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer;
impo