Flink和Kafka整合Demo以及DeserializationSchema.class找不到的解决方法

最新推荐文章于 2024-07-16 09:52:14 发布

泽米

最新推荐文章于 2024-07-16 09:52:14 发布

阅读量1.1w

点赞数 3

分类专栏： flink 文章标签： flink-kafk

本文链接：https://blog.csdn.net/a6822342/article/details/79035254

版权

flink 专栏收录该内容

8 篇文章 0 订阅

订阅专栏

这里用的是用官网提供的maven命令构建的flink1.4.0的flink-quick-start工程，具体构建工程命令如下

mvn archetype:generate -DarchetypeGroupId=org.apache.flink -DarchetypeArtifactId=flink-quickstart-java -DarchetypeVersion=1.4.0

然后会获取到一个java的初始工程。

下面是一个flink连接kafka的demo

import org.apache.flink.streaming.api.datastream.DataStream;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer010;
import org.apache.flink.streaming.util.serialization.SimpleStringSchema;
import org.apache.flink.streaming.util.serialization.DeserializationSchema;
import java.util.Properties;


public class connectkafka {
    private static String ZOOKEEPER_HOST="your-zookeeper-cluster";
    private static String KAFKA_BROKER="your-kafka-cluster";
    private static String GROUP="your-comsumer-group-name";

    public static void main(String[] args) throws Exception {


        StreamExecutionEnvironment env=StreamExecutionEnvironment.getExecutionEnvironment();

        connectkafka conn=new connectkafka();
        //test message:String. Real message :Tuple3<uid,itemid,rating>
        Properties kafkaprops=new Properties();
        kafkaprops.setProperty("zookeeper.connect",ZOOKEEPER_HOST);
        kafkaprops.setProperty("bootstrap.servers",KAFKA_BROKER);
        kafkaprops.setProperty("group.id",GROUP);

        DataStream<String> messagestream=env.addSource(new FlinkKafkaConsumer010<String>(
                "test",//这里是你的topic name
                new SimpleStringSchema(),
                kafkaprops));

        //show the message
        messagestream.print();
        env.execute();