flink 写入到es_Flink学习（十一） Sink到Elasticsearch

最新推荐文章于 2024-12-09 10:46:19 发布

weixin_39549936

最新推荐文章于 2024-12-09 10:46:19 发布

阅读量968

点赞数

文章标签： flink 写入到es

本文链接：https://blog.csdn.net/weixin_39549936/article/details/111884372

版权

本文展示了如何使用Apache Flink从文本文件读取数据，将其转换为`SensorReading`对象，并通过ElasticsearchSink写入到Elasticsearch。数据包括传感器ID、时间戳和温度值，Elasticsearch的索引名为`sensor`，类型为`readingdata`。

摘要由CSDN通过智能技术生成

packagecom.wyh.streamingApi.sinkimportjava.utilimportorg.apache.flink.api.common.functions.RuntimeContextimportorg.apache.flink.streaming.api.scala._importorg.apache.flink.streaming.connectors.elasticsearch.{ElasticsearchSinkFunction, RequestIndexer}importorg.apache.flink.streaming.connectors.elasticsearch6.ElasticsearchSinkimportorg.apache.http.HttpHostimportorg.elasticsearch.client.Requests//温度传感器读数样例类

case classSensorReading(id: String, timestamp: Long, temperature: Double)

object Sink2ES {

def main(args: Array[String]): Unit={

val env=StreamExecutionEnvironment.getExecutionEnvironment

env.setParallelism(1)//Source操作

val inputStream = env.readTextFile("F:\\flink-study\\wyhFlinkSD\\data\\sensor.txt")//Transform操作

val dataStream: DataStream[SensorReading] = inputStream.map(data =>{

val dataArray= data.split(",")</