kafka stream mysql_kafka stream 使用样例

1. war包导入

org.apache.kafka

kafka-streams

2.3.0

org.apache.kafka

kafka-clients

2.3.0

2. 代码实现

packagecom.atguigu.kafkastream;importorg.apache.kafka.common.serialization.Serde;importorg.apache.kafka.common.serialization.Serdes;importorg.apache.kafka.streams.KafkaStreams;importorg.apache.kafka.streams.StreamsBuilder;importorg.apache.kafka.streams.StreamsConfig;importorg.apache.kafka.streams.kstream.Consumed;importorg.apache.kafka.streams.kstream.KStream;importorg.apache.kafka.streams.kstream.Produced;importorg.apache.kafka.streams.processor.internals.InternalTopologyBuilder;importjava.util.Properties;public classApplication {public static voidmain(String[] args) {

String input= "abc"; //输入 topic

String output = "recommender"; //输出 topic

Properties properties= newProperties();

properties.put(StreamsConfig.APPLICATION_ID_CONFIG,"logProcessor");

properties.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG,"hadoop1:9092");//使用Serdes类创建序列化/反序列化所需的Serde实例 Serdes类为以下类型提供默认的实现：String、Byte array、Long、Integer和Double。

Serde stringSerde =Serdes.String();

StreamsBuilder builder= newStreamsBuilder();

KStream simpleFirstStream =builder.stream(input, Consumed.with(stringSerde, stringSerde));//使用KStream.mapValues 将输入数据流以 abc: 拆分获取下标为 1 字符串

KStream upperCasedStream = simpleFirstStream.mapValues(line -> line.split("abc:")[1]);//把转换结果输出到另一个topic

upperCasedStream.to(output, Produced.with(stringSerde, stringSerde));//创建和启动KStream

KafkaStreams kafkaStreams = newKafkaStreams(builder.build(), properties);

kafkaStreams.start();

}

}

3. 测试

1)启动 2 中程序

2)启动 kafka

3)启动一个名称为 abc 的 topic 生产者

bin/kafka-topics.sh --create --zookeeper hadoop1:2181 --replication-factor 1 --partitions 1 --topic abc

4)启动一个名词为 recommender 的topic 消费者

bin/kafka-console-consumer.sh --bootstrap-server hadoop1:9092 --topic recommender

5)abc topic 中输入字符串(如：   abc:22|33|44|55)

6)recommender 中就可收到过滤后的字符串   22|33|44|55