SpringCloud框架集成zookeeper集群+kafka集群

1.创建父子级项目结构,生产者springboot微服务与消费者springboot微服务。

2.生产者创建一个供与测试的实体Bean

package com.jk.bean;

import java.util.Date;

public class MessageBean {

    private Long id;    //id
    private String msg; //消息
    private Date sendTime;  //时间戳

    @Override
    public String toString() {
        return "MessageBean{" +
                "id=" + id +
                ", msg='" + msg + '\'' +
                ", sendTime=" + sendTime +
                '}';
    }

    public Long getId() {
        return id;
    }

    public void setId(Long id) {
        this.id = id;
    }

    public String getMsg() {
        return msg;
    }

    public void setMsg(String msg) {
        this.msg = msg;
    }

    public Date getSendTime() {
        return sendTime;
    }

    public void setSendTime(Date sendTime) {
        this.sendTime = sendTime;
    }
}

3.生产者创建controller层,创建producer类

   创建一个Gson对象 提供java对象和json字符串之间进行转化

package com.jk.controller;

import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import com.jk.bean.MessageBean;
import lombok.extern.slf4j.Slf4j;
import org.junit.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.stereotype.Component;

import java.util.Date;
import java.util.UUID;

@Component
@Slf4j
public class producer {

   @Autowired
   KafkaTemplate<String,String> kafkaTemplate;

   //创建一个Gson对象 提供java对象和json字符串之间进行转化
   private Gson gson = new GsonBuilder().create();


   public void send() {
   MessageBean message = new MessageBean();
   message.setId(System.currentTimeMillis());
   message.setMsg(UUID.randomUUID().toString());
   message.setSendTime(new Date());

     //topic-ideal为主题
      //send方法 发送消息
   kafkaTemplate.send("test", gson.toJson(message));
     }

}

4.controller层下创建测试启动类写一个测试方法:

package com.jk.controller;

import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;

@RunWith(SpringRunner.class)
@SpringBootTest
public class KafkaProducerApplicationTests{
    @Autowired
    private producer kafkaProducer;
    @Test
    public void kafkaProducer(){
        this.kafkaProducer.send();
    }
    @Test
    public void contextLoads() {
    }
}

5.配置yml配置文件


spring:
  kafka:
     bootstrap-servers: 192.168.1.120:9092,192.168.1.120:9093,192.168.1.120:9094
     producer:

       retries: 0


       batch-size: 16384

       buffer-memory: 33554432

       key-serializer: org.apache.kafka.common.serialization.StringSerializer
       value-serializer: org.apache.kafka.common.serialization.StringSerializer

bootstrap-servers: 是自己linux系统ip+kafka端口号,有几个写几个

6.消费者创建controller层,创建KafkaConsumer类接受消息

package com.jk.controller;

import lombok.extern.slf4j.Slf4j;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.stereotype.Component;

import java.util.Optional;

import static org.apache.kafka.common.requests.DeleteAclsResponse.log;

@Component
@Slf4j
public class KafkaConsumer{

    @KafkaListener(topics = {"test"})
   public void consumer(ConsumerRecord<?, ?> record){
   Optional<?> kafkaMessage = Optional.ofNullable(record.value());
   if (kafkaMessage.isPresent()) {
   Object message = kafkaMessage.get();
   log.info("----------------- record =" + record);
   log.info("------------------ message =" + message);
    }
    }
}

7.配置消费者yml配置文件,与生产者配置基本一致:

server:
  port: 9999
spring:
  kafka:
    bootstrap-servers: 192.168.1.120:9093,192.168.1.120:9094,192.168.1.120:9092
    consumer:
      group-id: ideal-consumer-group
      auto-offset-reset: earliest
      enable-auto-commit: true
      auto-commit-interval: 20000
      key-deserializer: org.apache.kafka.common.serialization.StringDeserializer
      value-deserializer: org.apache.kafka.common.serialization.StringDeserializer

8.启动生产者测试方法send

9.启动消费者xshell将会看到消息接收到了

完成.

附上测试项目下载地址:

本项目仅供下载学习:

链接:https://pan.baidu.com/s/1z9a2JDJKpLPL-0lP-4w3Cw 
提取码:vf3t 

 

 

  • 1
    点赞
  • 5
    收藏
    觉得还不错? 一键收藏
  • 打赏
    打赏
  • 4
    评论
好的,以下是一个基本的在k8s上部署zookeeper + kafka集群的配置教程: 1. 创建一个namespace 首先,我们需要创建一个namespace,用于部署zookeeperkafka集群。可以使用以下命令创建一个名为“zookeeper-kafka”的namespace: ``` kubectl create namespace zookeeper-kafka ``` 2. 部署Zookeeper 接下来,我们需要部署Zookeeper。可以使用以下YAML文件创建一个Zookeeper服务: ``` apiVersion: v1 kind: Service metadata: name: zookeeper spec: selector: app: zookeeper ports: - name: client port: 2181 protocol: TCP - name: follower port: 2888 protocol: TCP - name: leader port: 3888 protocol: TCP --- apiVersion: apps/v1 kind: StatefulSet metadata: name: zookeeper spec: serviceName: zookeeper replicas: 3 selector: matchLabels: app: zookeeper template: metadata: labels: app: zookeeper spec: containers: - name: zookeeper image: zookeeper:3.5.8 ports: - name: client containerPort: 2181 - name: follower containerPort: 2888 - name: leader containerPort: 3888 volumeMounts: - name: datadir mountPath: /data env: - name: ZOO_MY_ID valueFrom: fieldRef: fieldPath: metadata.name - name: ZOO_SERVERS value: zookeeper-0.zookeeper:2888:3888,zookeeper-1.zookeeper:2888:3888,zookeeper-2.zookeeper:2888:3888 volumeClaimTemplates: - metadata: name: datadir spec: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi ``` 这将创建一个3个Pod的Zookeeper StatefulSet,并创建一个名为“zookeeper”的Service,暴露Zookeeper的客户端端口2181,follower端口2888和leader端口3888。 3. 部署Kafka 现在,我们可以部署Kafka。以下是一个Kafka部署的YAML文件示例: ``` apiVersion: v1 kind: Service metadata: name: kafka spec: type: NodePort selector: app: kafka ports: - name: kafka port: 9092 nodePort: 30092 protocol: TCP --- apiVersion: apps/v1 kind: StatefulSet metadata: name: kafka spec: serviceName: kafka replicas: 3 selector: matchLabels: app: kafka template: metadata: labels: app: kafka spec: containers: - name: kafka image: wurstmeister/kafka:2.13-2.7.0 ports: - name: kafka containerPort: 9092 env: - name: KAFKA_BROKER_ID valueFrom: fieldRef: fieldPath: metadata.name - name: KAFKA_ZOOKEEPER_CONNECT value: zookeeper-0.zookeeper:2181,zookeeper-1.zookeeper:2181,zookeeper-2.zookeeper:2181 - name: KAFKA_ADVERTISED_LISTENERS value: PLAINTEXT://$(hostname -f):9092 - name: KAFKA_LISTENERS value: PLAINTEXT://0.0.0.0:9092 - name: KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR value: "3" volumeMounts: - name: datadir mountPath: /data volumeClaimTemplates: - metadata: name: datadir spec: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi ``` 这将创建一个3个Pod的Kafka StatefulSet和一个名为“kafka”的Service,它将Kafka的9092端口暴露为NodePort 30092。 4. 验证部署 现在,您可以使用以下命令检查ZookeeperKafka是否正在运行: ``` kubectl get pods -n zookeeper-kafka ``` 您应该看到3个Zookeeper和3个Kafka Pod处于“Running”状态。 接下来,您可以使用以下命令检查Kafka是否正在监听端口30092(或您自己选择的端口): ``` kubectl get services -n zookeeper-kafka ``` 您应该看到一个名为“kafka”的service,它将Kafka的9092端口暴露为30092端口。可以使用此端口测试Kafka是否正常运行。 至此,您已经成功地在k8s上部署了zookeeper + kafka集群

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 4
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

听风动

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值