java afka_java-如何将两个Kafka流结合在一起,并在具有...

我有两个Kafka Streams,它们具有String键和我使用KSQL创建的Avro格式的值.

这是第一个:

DESCRIBE EXTENDED STREAM_1;

Type : STREAM

Key field : IDUSER

Timestamp field : Not set - using

Key format : STRING

Value format : AVRO

Kafka output topic : STREAM_1 (partitions: 4, replication: 1)

Field | Type

--------------------------------------------------------

ROWTIME | BIGINT (system)

ROWKEY | VARCHAR(STRING) (system)

FIRSTNAME | VARCHAR(STRING)

LASTNAME | VARCHAR(STRING)

IDUSER | VARCHAR(STRING)

第二个:

DESCRIBE EXTENDED STREAM_2;

Type : STREAM

Key field : IDUSER

Timestamp field : Not set - using

Key format : STRING

Value format : AVRO

Kafka output topic : STREAM_2 (partitions: 4, replication: 1)

Field | Type

--------------------------------------------------------

ROWTIME | BIGINT (system)

ROWKEY | VARCHAR(STRING) (system)

USERNAME | VARCHAR(STRING)

IDUSER | VARCHAR(STRING)

DEVICE | VARCHAR(STRING)

所需的输出应包括IDUSER,LASTNAME,DEVICE和USERNAME.

我想使用Streams API离开(在IDUSER上)加入这些流,并将输出写入kafka主题.

为此,我尝试了以下方法:

public static void main(String[] args) {

final Properties streamsConfiguration = new Properties();

streamsConfiguration.put(StreamsConfig.APPLICATION_ID_CONFIG, "kafka-strteams");

streamsConfiguration.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");

streamsConfiguration.put(StreamsConfig.ZOOKEEPER_CONNECT_CONFIG, "localhost:2181");

streamsConfiguration.put(AbstractKafkaAvroSerDeConfig.SCHEMA_REGISTRY_URL_CONFIG, "http://localhost:8081");

streamsConfiguration.put(StreamsConfig.KEY_SERDE_CLASS_CONFIG, Serdes.String().getClass().getName());

streamsConfiguration.put(StreamsConfig.VALUE_SERDE_CLASS_CONFIG, GenericAvroSerde.class);

streamsConfiguration.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");

final Serde stringSerde = Serdes.String();

final Serde genericAvroSerde = new GenericAvroSerde();

boolean isKeySerde = false;

genericAvroSerde.configure(Collections.singletonMap(AbstractKafkaAvroSerDeConfig.SCHEMA_REGISTRY_URL_CONFIG, "http://localhost:8081"), isKeySerde);

KStreamBuilder builder = new KStreamBuilder();

KStream left = builder.stream("STREAM_1");

KStream right = builder.stram("STREAM_2");

// Java 8+ example, using lambda expressions

KStream joined = left.leftJoin(right,

(leftValue, rightValue) -> "left=" + leftValue + ", right=" + rightValue, /* ValueJoiner */

JoinWindows.of(TimeUnit.MINUTES.toMillis(5)),

Joined.with(

stringSerde, /* key */

genericAvroSerde, /* left value */

genericAvroSerde) /* right value */

);

joined.to(stringSerde, genericAvroSerde, "streams-output-testing");

KafkaStreams streams = new KafkaStreams(builder, streamsConfiguration);

streams.cleanUp();

streams.start();

Runtime.getRuntime().addShutdownHook(new Thread(streams::close));

}

然而,

KStream joined = ...

在我的IDE上抛出错误:

incompatible types: inference variable VR has incompatible bounds

当我尝试将String Serde用作键和值时,它可以工作,但数据不能从kafka-console-consumer读取.我要做的是以AVRO格式生成数据,以便能够使用kafka-avro-console-consumer读取它们.

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值