在kafka中,我想指定分区的偏移量开始消费
KafkaConsumer<Integer,String> consumer = new KafkaConsumer(props);
consumer.subscribe(Arrays.asList("test"));
consumer.seek(new TopicPartition("test",0),0l);
Exception in thread "main" java.lang.IllegalStateException: No current assignment for partition test-0
at org.apache.kafka.clients.consumer.internals.SubscriptionState.assignedState(SubscriptionState.java:268)
at org.apache.kafka.clients.consumer.internals.SubscriptionState.seek(SubscriptionState.java:293)
at org.apache.kafka.clients.consumer.KafkaConsumer.seek(KafkaConsumer.java:1139)
at KafkaConsumerDemo.main(KafkaConsumerDemo.java:25)
google一下
https://stackoverflow.com/questions/41008610/kafkaconsumer-0-10-java-api-error-message-no-current-assignment-for-partition
大意是说要先poll一次才可以.
KafkaConsumer<Integer,String> consumer = new KafkaConsumer(props);
consumer.subscribe(Arrays.asList("test"));
ConsumerRecords<Integer, String> recordTemp = consumer.poll(0);
System.out.println(recordTemp.isEmpty());
consumer.seek(new TopicPartition("test",0),0l);
这样确实解决了问题,从报错的原因可以看出
/* the list of partitions currently assigned */
//无论使用什么订阅模式,都使用此集合记录每个topic的消费状态
private final Map<TopicPartition, TopicPartitionState> assignment;
是上面这个字段导致的,只有第一次启动才会导致这个问题. poll方法又拉取数据,又帮忙初始化元数据.下面分析一下,第一次poll,它有没有去拉数据,干了什么?
通过阅读代码,发现其实第一次poll也拉取了数据,不过由于偏移量不是我们想要的,就被丢弃了
// we are interested in this fetch only if the beginning offset matches the
// current consumed position
//仅仅当开的偏移量 匹配当前消费的位置,我们才对这次拉取有兴趣
Long position = subscriptions.position(tp);
if (position == null || position != fetchOffset) {
log.debug("Discarding stale fetch response for partition {} since its offset {} does not match " +
"the expected offset {}", tp, fetchOffset, position);
return null;
}
同时也可以看到日志
[2018-08-24 11:46:24,273] DEBUG Discarding stale fetch response for partition testBug-0 since its offset 61898 does not match the expected offset 10 (org.apache.kafka.clients.consumer.internals.Fetcher)