kafka从头消费信息

命令行

1 group id换成新的
2 offset参数设为earliest

kafka-console-consumer.sh \
--bootstrap-server mypc01:9092,mypc02:9092,mypc03:9092 \
--topic cat \
--partition 0 \
--group group1 \
--offset earliest

代码

seekToBeginning方法可以让从头开始消费

//从头开始消费
  def ConsumerTest3(): Unit = {
    val properties = new Properties()
    val stream: InputStream = ConsumerDemo2.getClass.getClassLoader.getResourceAsStream("consumer.properties")
    properties.load(stream)
    val consumer = new KafkaConsumer[String, String](properties)
    //指定需要消费的分区
    val partition1 = new TopicPartition("cat", 0)
    //必须设置,f,否则报错Consumer is not subscribed to any topics or assigned any partitions
    consumer.assign(Arrays.asList(partition1))
    import scala.collection.JavaConversions._
    consumer.poll(1000)
    //从头开始
    consumer.seekToBeginning(Arrays.asList(partition1))
    while (true) {
      val records: ConsumerRecords[String, String] = consumer.poll(1000)
      for (x <- records) {
        println(x)
      }
    }
  }

结果 可以看到 输出的第一条消息offset为0

ConsumerRecord(topic = cat, partition = 0, offset = 0, CreateTime =

当然,如下写法也是可以从头消费的

 consumer.seek(partition1,0)

总结

• kafka从头消费需要使用seek方法或者seekToBeginning方法