java afka_java-如何将两个Kafka流结合在一起,并在具有...

我有两个Kafka Streams,它们具有String键和我使用KSQL创建的Avro格式的值.

这是第一个：

DESCRIBE EXTENDED STREAM_1;

Type : STREAM

Key field : IDUSER

Timestamp field : Not set - using

Key format : STRING

Value format : AVRO

Kafka output topic : STREAM_1 (partitions: 4, replication: 1)

Field | Type

--------------------------------------------------------

ROWTIME | BIGINT (system)

ROWKEY | VARCHAR(STRING) (system)

FIRSTNAME | VARCHAR(STRING)

LASTNAME | VARCHAR(STRING)

IDUSER | VARCHAR(STRING)

第二个：

DESCRIBE EXTENDED STREAM_2;

Type : STREAM

Key field : IDUSER

Timestamp field : Not set - using

Key format : STRING

Value format : AVRO

Kafka output topic : STREAM_2 (partitions: 4, replication: 1)

Field | Type

--------------------------------------------------------

ROWTIME | BIGINT (system)

ROWKEY | VARCHAR(STRING) (system)

USERNAME | VARCHAR(STRING)

IDUSER | VARCHAR(STRING)

DEVICE | VARCHAR(STRING)

所需的输出应包括IDUSER,LASTNAME,DEVICE和USERNAME.

我想使用Streams API离开(在IDUSER上)加入这些流,并将输出写入kafka主题.

为此,我尝试了以下方法：

public static void main(String[] args) {

final Properties streamsConfiguration = new Properties();

streamsConfiguration.put(StreamsConfig.APPLICATION_ID_CONFIG, "kafka-strteams");

streamsConfiguration.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");

streamsConfiguration.put(StreamsConfig.ZOOKEEPER_CONNECT_CONFIG, "localhost:2181");

streamsConfiguration.put(AbstractKafkaAvroSerDeConfig.SCHEMA_REGISTRY_URL_CONFIG, "http://localhost:8081");

streamsConfiguration.put(StreamsConfig.KEY_SERDE_CLASS_CONFIG, Serdes.String().getClass().getName());

streamsConfiguration.put(StreamsConfig.VALUE_SERDE_CLASS_CONFIG, GenericAvroSerde.class);

streamsConfiguration.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");

final Serde stringSerde = Serdes.String();

final Serde genericAvroSerde = new GenericAvroSerde();

boolean isKeySerde = false;

genericAvroSerde.configure(Collections.singletonMap(AbstractKafkaAvroSerDeConfig.SCHEMA_REGISTRY_URL_CONFIG, "http://localhost:8081"), isKeySerde);

KStreamBuilder builder = new KStreamBuilder();

KStream left = builder.stream("STREAM_1");

KStream right = builder.stram("STREAM_2");

// Java 8+ example, using lambda expressions

KStream joined = left.leftJoin(right,

(leftValue, rightValue) -> "left=" + leftValue + ", right=" + rightValue, /* ValueJoiner */

JoinWindows.of(TimeUnit.MINUTES.toMillis(5)),

Joined.with(

stringSerde, /* key */

genericAvroSerde, /* left value */

genericAvroSerde) /* right value */

);

joined.to(stringSerde, genericAvroSerde, "streams-output-testing");

KafkaStreams streams = new KafkaStreams(builder, streamsConfiguration);

streams.cleanUp();

streams.start();

Runtime.getRuntime().addShutdownHook(new Thread(streams::close));

}

然而,

KStream joined = ...

在我的IDE上抛出错误：

incompatible types: inference variable VR has incompatible bounds

当我尝试将String Serde用作键和值时,它可以工作,但数据不能从kafka-console-consumer读取.我要做的是以AVRO格式生成数据,以便能够使用kafka-avro-console-consumer读取它们.