kafka 订阅模式 java,Spring Kafka：在运行时订阅新的主题模式

I'm using the annotation @KafkaListener to consume topics in my application. I need to change the topic pattern at runtime in an already running consumer so that new topics that match the new pattern can be consumed.

I tried the below code, but it still consumes the topics matching the old topic pattern. Here, I have set the "old-topic-pattern" at application start-up. Then, I'm updating the pattern to "new-topic-pattern" every 10 seconds using a Spring @Scheduler.

Class "KafkaTopicPatternConfig.java":

@Configuration

public class KafkaTopicPatternConfig {

@Bean

public String kafkaTopicPattern(Environment env) {

logger.info("Getting kafka topic pattern");

String kafkaTopicPattern = "old-topic-pattern";

return kafkaTopicPattern;

}

}

Class "Consumer.java":

@Component

public class Consumer implements ConsumerSeekAware{

@Autowired

@Qualifier("kafkaTopicPattern")

private String kafkaTopicPattern;

@KafkaListener(topicPattern = "#{kafkaTopicPattern}", id = "s4federatorConsumer")

public void processMessage(@Payload ConsumerRecord record,

@Header(KafkaHeaders.OFFSET) Long offset,

@Header(KafkaHeaders.CONSUMER) KafkaConsumer consumer,

@Header(KafkaHeaders.RECEIVED_PARTITION_ID) Integer partitionId) {

//do something with the consumed message

}

@Scheduled(fixedDelay = 10000, initialDelay = 15000)

public void refreshKafkaTopics() {

logger.info("Inside scheduler to refresh kafka topics");

this.kafkaTopicPattern = "new-topic-pattern";

this.kafkaListenerEndpointRegistry.getListenerContainer("s4federatorConsumer").stop();

this.kafkaListenerEndpointRegistry.getListenerContainer("s4federatorConsumer").start();

}

}

解决方案

You are getting kafkaTopicPattern as -

@Qualifier("kafkaTopicPattern")

private String kafkaTopicPattern;

I see you are updating the pattern like -

this.kafkaTopicPattern = "new-topic-pattern";

But the original value for "kafkaTopicPattern" which is injected in listener wont be refreshed by this if these 2 are in different instance objects. So you will have to make sure that the listener objects are refreshed with the new pattern.