kafka的安装与使用

一、kafka的单机版安装

kafka的使用需要依赖于zookeeper，zookeeper的安装过程见zookeeper的安装与使用

1.1 kafka的安装

将kafka的安装包上传到Linux系统，并解压。

1.2 修改配置文件

1.3 kafka启动

bin/kafka-server-start.sh config/server.properties

二、kafka的使用

2.1 创建topic

bin/kafka-topics.sh --zookeeper localhost:2181 --create --topic heima --partitions 2 --replication-factor 1
使用bin下的创建topic命令，指定zookeeper的地址，topic的名称，分区数和副本数

2.2 查看所有创建的主题

bin/kafka-topics.sh --zookeeper localhost:2181 --list

2.3 查看主题的详情

bin/kafka-topics.sh --zookeeper localhost:2181 --describe --topic heima

2.4 消费者接收消息

bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic heima

2.5 生产者生产消息

bin/kafka-console-producer.sh --broker-list localhost:9092 --topic heima

2.6 简单实现Java操作kafka

2.6.1 生产者生产消息

public class ProducerFastStart {

    public static final String brokerList = "192.168.58.129:9092";
    public static final String topic = "heima";

    public static void main(String[] args) {
        Properties properties = new Properties();
        // 设置key序列化器
        //properties.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
        properties.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
        //设置重试次数
        properties.put(ProducerConfig.RETRIES_CONFIG, 1);
        // 设置值序列化器
        //properties.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
        properties.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG,StringSerializer.class.getName());
        //设置集群地址
        //properties.put("bootstrap.servers", brokerList);
        properties.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG,brokerList);

        KafkaProducer<String, String> producer = new KafkaProducer<String, String>(properties);
        ProducerRecord<String, String> record = new ProducerRecord<>(topic, "kafka-demo", "hello,kafka!");
        try {
            producer.send(record);
        } catch (Exception e) {
            e.printStackTrace();
        }
        producer.close();
    }
}

2.6.2 消费者消费消息

public class ConsumerFastStart {

    public static final String brokerList = "192.168.58.129:9092";
    public static final String topic = "heima";
    public static final String groupId = "group.demo";

    public static void main(String[] args) {
        Properties properties = new Properties();
        // 设置key序列化器
        //properties.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
        properties.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName());
        // 设置值序列化器
        //properties.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
        properties.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName());
        //设置集群地址
        //properties.put("bootstrap.servers", brokerList);
        properties.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG,brokerList);
        //properties.put("group.id", groupId);
        properties.put(ConsumerConfig.GROUP_ID_CONFIG,groupId);

        KafkaConsumer<String, String> consumer = new KafkaConsumer<String, String>(properties);
        consumer.subscribe(Collections.singletonList(topic));
        while (true) {
            ConsumerRecords<String,String> records = consumer.poll(Duration.ofMillis(1000));
            for (ConsumerRecord<String, String> record : records) {
                System.out.println(record.value());
            }
        }
    }
}

使用时虚拟的防火墙一定要关闭
查看防火墙状态命令：firewall-cmd --state
关闭防火墙命令：systemctl stop firewalld.service