Win10下kafka的下载安装启动以及用java代码操作kafka

kafka依赖于zookeeper，现在官网版本呢的kafka内置了zookeeper

一.去kafka官网下载相应的kafka版本

下载完成之后进行解压，然后进入到kafka根目录

在kafka根目录的文件夹下创建两个文件夹data和kafka-logs。用来作为kafka数据存储以及日志记录文件夹

然后分别修改上面的两个文件

修改server.properties中的log.dirs为你创建的kafka-logs目录

修改zookeeper.properties中的dataDir为你刚才创建的data文件夹路径

二.启动

启动时进入到kafka的根目录下
由于kafka依赖zookeeper，所以必须先启动zookeeper，在启动zookeeper以及kafka时都需要带上各自的配置文件

启动zookeeper执行下面的命令，需要注意的时kafka不要放的文件目录太深，要不然会报错
.\bin\windows\zookeeper-server-start.bat .\config\zookeeper.properties

启动成功不关闭

启动kafka
.\bin\windows\kafka-server-start.bat .\config\server.properties

启动成功不关闭

Java代码操作kafka

引入依赖

<dependency>
      <groupId>org.apache.kafka</groupId>
      <artifactId>kafka-clients</artifactId>
      <version>2.1.0</version>
    </dependency>

生产者

import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerConfig;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.apache.kafka.common.serialization.StringSerializer;

import java.util.Properties;
import java.util.Random;

public class Producer {
    public static String topic = "duanjt_test";//定义主题

    public static void main(String[] args) throws InterruptedException {
        Properties p = new Properties();
        p.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "0.0.0.0:9092");//kafka地址，多个地址用逗号分割
        p.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
        p.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
        KafkaProducer<String, String> kafkaProducer = new KafkaProducer<>(p);

        try {
            while (true) {
                String msg = "Hello," + new Random().nextInt(100);
                ProducerRecord<String, String> record = new ProducerRecord<String, String>(topic, msg);
                kafkaProducer.send(record);
                System.out.println("消息发送成功:" + msg);
                Thread.sleep(500);
            }
        } finally {
            kafkaProducer.close();
        }

    }
}

消费者

import org.apache.kafka.clients.consumer.ConsumerConfig;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.apache.kafka.clients.consumer.ConsumerRecords;
import org.apache.kafka.clients.consumer.KafkaConsumer;
import org.apache.kafka.common.serialization.StringDeserializer;

import java.time.Duration;
import java.util.Collections;
import java.util.Properties;

public class Consumer {
    public static void main(String[] args) {
        Properties p = new Properties();
        p.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "0.0.0.0:9092");
        p.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
        p.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
        p.put(ConsumerConfig.GROUP_ID_CONFIG, "duanjt_test");

        KafkaConsumer<String, String> kafkaConsumer = new KafkaConsumer<String, String>(p);
        kafkaConsumer.subscribe(Collections.singletonList(Producer.topic));// 订阅消息

        while (true) {
            // 每隔一秒读取Kafka数据
            ConsumerRecords<String, String> records = kafkaConsumer.poll(Duration.ofSeconds(1));
            for (ConsumerRecord<String, String> red : records) {
                System.out.printf("offset:%d,key:%s,value:%s", red.offset(), red.key(), red.value());
            }
        }
    }
}

在执行时先启动消费者代码，在启动生产者代码

执行结果

生产者控制台打印

消费者控制台打印

可以看到生产者生产的数据，消费者都捕获到了，并且打印在控制台