kafka学习（四）--java开发(基于kafka0.9、1.0版本)

kafka-clients jar包采用的是org.apache.kafka:kafka-clients:0.9.0.1。

Producer

Properties props = new Properties();
props.put("bootstrap.servers", "x.x.x.x:9091");
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("acks", "all");
props.put("retries", 1);

Producer<String, String> producer = new KafkaProducer<String, String>(props);
producer.send(new ProducerRecord<String, String>(topic, "Hello"));

producer.send(new ProducerRecord<String, String>(topic, "World"), new Callback() {
    @Override
    public void onCompletion(RecordMetadata metadata, Exception e) {
        if (e != null) {
            e.printStackTrace();
        } else {
            System.out.println(metadata.toString());//org.apache.kafka.clients.producer.RecordMetadata@1d89e2b5
            System.out.println(metadata.offset());//1
        }
    }
});
producer.flush();
producer.close();

Consumer

Properties props = new Properties();
props.put("bootstrap.servers", " x.x.x.x:9091");
props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
props.setProperty("group.id", "0");
props.setProperty("enable.auto.commit", "true");
props.setProperty("auto.offset.reset", "earliest");

Consumer<String, String> consumer = new KafkaConsumer<String, String>(props);
consumer.subscribe(Lists.newArrayList(topic));

for (int i = 0; i < 2; i++) {
    ConsumerRecords<String, String> records = consumer.poll(1000);
    System.out.println(records.count());
    for (ConsumerRecord<String, String> record : records) {
        System.out.println(record);
        //consumer.seekToBeginning(new TopicPartition(record.topic(), record.partition()));
    }
}

注意，auto.offset.reset这里设置为earliest，是为了consumer能够从头开始读取内容即offset=0开始,在org.apache.kafka.clients.consumer.ConsumerConfig中对其意义的描述如下：What to do when there is no initial offset in Kafka or if the current offset does not exist any more on the server (e.g. because that data has been deleted): earliest: automatically reset the offset to the earliest offset； latest: automatically reset the offset to the latest offset； none: throw exception to the consumer if no previous offset is found for the consumer's group； anything else: throw exception to the consumer。consumer.seekToBeginning也可以设置offset，但是跟源码发现，This function evaluates lazily, seeking to the final offset in all partitions only when {@link #poll(long)} or {@link #position(TopicPartition)} are called.也就是说seekToBeginning()的设置要生效的话，必须在poll或则position方法调用后设置seekToBeginning()才行。