kafka 测试案例

一.启动自带的zookeeper

1. bin/zookeeper-server-start.sh config/zookeeper.properties
2. 可以打开zookeeper中的 zkCli.sh 查看节点

二.启动kafaka

1. bin/kafka-server-start.sh config/server.properties
2. 注意: 如果使用 & 在后台启动 则bin/kafka-server-stop.sh关闭不了 需要通过jobs 查看 后 kill %jobnum关闭

三.创建topic

1. bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
2. 查看topic : bin/kafka-topics.sh --list --zookeeper localhost:2181

四.生产者发送消息

1. bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test

发送 adfadadfadf

五.消费者接受消息

1. bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test

显示 adfadadfadf

六.java引入kafka

1. 添加kafka依赖 (注意:不需要添加zookeeper依赖)
2. 配置application.properties文件

#kafka的ip
spring.kafka.bootstrap-servers=localhost:9092   
#消费者的 key value 及接受消息ip
spring.kafka.consumer.bootstrap-servers=localhost:9092
spring.kafka.consumer.group-id=kafka2
spring.kafka.consumer.key-deserializer=org.apache.kafka.common.serialization.StringDeserializer
spring.kafka.consumer.value-deserializer=org.apache.kafka.common.serialization.StringDeserializer
#生产值的key value格式 及发布地址
spring.kafka.producer.bootstrap-servers=localhost:9092
spring.kafka.producer.key-serializer=org.apache.kafka.common.serialization.StringSerializer
spring.kafka.producer.value-serializer=org.apache.kafka.common.serialization.StringSerializer

3. 发送消息:使用注入kafkaTemplate 调用 send()方法
4. 接受消息: 添加@KafkaListener(topics = “test”)注解 传出(ConsumerRecord<?,?> record)参数 调用record的 key(),value()方法

实例如下

1. pom依赖

  <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.1.5.RELEASE</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>

  <dependency>
         <groupId>org.springframework.kafka</groupId>
         <artifactId>spring-kafka</artifactId>
  </dependency>

2. application.properties

spring.kafka.bootstrap-servers=192.168.1.106:9092
#spring.kafka.consumer.bootstrap-servers=localhost:9092
spring.kafka.consumer.group-id=kafka2
spring.kafka.consumer.key-deserializer=org.apache.kafka.common.serialization.StringDeserializer
spring.kafka.consumer.value-deserializer=org.apache.kafka.common.serialization.StringDeserializer
#spring.kafka.producer.bootstrap-servers=localhost:9092
spring.kafka.producer.key-serializer=org.apache.kafka.common.serialization.StringSerializer
spring.kafka.producer.value-serializer=org.apache.kafka.common.serialization.StringSerializer

3. java类

// 发送消息类
@Component
public class Test {
    @Autowired
    KafkaTemplate kafkaTemplate;
    @Scheduled(fixedDelay = 10000)
    public void sendMessage(){
        kafkaTemplate.send("test","你好 我是kafka自动任务");
        System.out.println("success");
    }
}

//  接收消息类
@Component
public class KafkaMessage {
    @Autowired
    MessageRepository messageRepository;

    @KafkaListener(topics = "test")
    public void saveMessage(ConsumerRecord<?,?> record){
        String value = (String)record.value();
        Message message=new Message(System.currentTimeMillis(),"f_kafka","t_es",value);
        messageRepository.save(message);
    }
}

在linux部署

1. 首先需要关闭防火墙 chkconfig iptables off(on) 永久 即刻生效: service iptables stop(start)
2. 同时需要修改 etc/hosts 127.0.0.1 后增加你的主机名re
3. 配置 kafka /config/server.properties 修改 listeners=PLAINTEXT://192.168.1.106:9092(外网能访问)
4. 创建topic bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
5. 修改程序的application.properties

spring.kafka.bootstrap-servers=192.168.1.106:9092   
spring.kafka.consumer.bootstrap-servers=192.168.1.106:9092
spring.kafka.producer.bootstrap-servers=192.168.1.106:9092