启动消费线程提示错误:
WARN 2020-02-25 00:16:47 com.spider.base.kafka.task.MyKafkaConsumerTask.run(45) | 任务执行异常 topic:data_syn_topic e:kafka.common.MessageStreamsExistException: ZookeeperConsumerConnector can create message streams at most once
at kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreams(ZookeeperConsumerConnector.scala:79)
at kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreams(ZookeeperConsumerConnector.scala:96)
at com.spider.base.kafka.util.MyKafkaContext.getKafkaStreamListByTopic(MyKafkaContext.java:45)
at com.spider.base.kafka.task.MyKafkaConsumerTask.run(MyKafkaConsumerTask.java:33)
at java.lang.Thread.run(Thread.java:748)
解决方法:
ConsumerConnector consumerConnector = Consumer.createJavaConsumerConnector(new ConsumerConfig(CcKafkaPropertiesUtil.getConsumerConfig()));
每个不同的topic都新生成一个consumerConnector
错误都写法:
/** 错误都写法:获取消费者列表 */
public static ConsumerConnector getConsumer(){
if(consumerConnector == null) {
consumerConnector = Consumer.createJavaConsumerConnector(new ConsumerConfig(CcKafkaPropertiesUtil.getConsumerConfig()));
}
return consumerConnector;
}