Flink和Kafka scala版本_flink1.14 kafkasource java可以正常编译 scala无法重载-CSDN博客

本文链接：https://blog.csdn.net/wangchao_cn/article/details/104707357

该博客展示了如何在Scala中使用Flink处理Kafka数据流。通过创建DataStream源来消费Kafka，并使用FlinkKafkaProducer进行序列化和反序列化。同时，文章还演示了从MySQL数据库读取数据并转换为Flink DataStream，然后将处理后的数据写入到Kafka。

摘要由CSDN通过智能技术生成

使用DataStream来消费Kafka。

下面Schema类很重要，实现对消息的序列化和反序列化

KafkaSerializationSchema<Quotes>, KafkaDeserializationSchema<Quotes>

import java.util.Properties

import com.redblue.entity.Quotes
import com.redblue.kafka.QuotesSchema
import org.apache.flink.api.common.functions.MapFunction
import org.apache.flink.api.common.typeinfo.BasicTypeInfo
import org.apache.flink.api.java.io.jdbc.JDBCInputFormat
import org.apache.flink.api.java.typeutils.RowTypeInfo
import org.apache.flink.streaming.api.datastream.DataStreamSource
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment
import org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer
import org.apache.flink.table.api.EnvironmentSettings
import org.apache.flink.table.api.java.StreamTableEnvironment
import org.apache.flink.types.Row

import scala.beans.BeanProperty

object TestKafka {
def main(args: Array[String]): Unit = {

val envSettings = EnvironmentSettings.newInstance.useOldPlanner.inStreamingMode.build
val env = StreamExecutionEnvironment.getExecutionEnvironment
val tableEnv = StreamTableEnvironment.create(env, envSettings)

val mysqlUrl = "jdbc:mysql://xxx.com:3306/test?useSSL=false"
val driverName = "com.mysql.jdbc.Driver"
val user = "root"
val password = "xxx"
val properties = new Properties()
properties.setProperty("user", user)
properties.setProperty("password", password)
val tableName = "stock_quotes"