Kafka： ------ 生产者、消费者、序列化、自定义分区。

最新推荐文章于 2021-11-21 15:42:25 发布
最新推荐文章于 2021-11-21 15:42:25 发布
阅读量140 收藏
点赞数
分类专栏： Kafka 文章标签： kafka
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/weixin_45764675/article/details/105477698
版权

Kafka API实战

依赖

  <dependency>
      <groupId>org.apache.kafka</groupId>
      <artifactId>kafka-clients</artifactId>
      <version>2.2.0</version>
    </dependency>

    <!-- https://mvnrepository.com/artifact/log4j/log4j -->
    <dependency>
      <groupId>log4j</groupId>
      <artifactId>log4j</artifactId>
      <version>1.2.17</version>
    </dependency>
    <!-- https://mvnrepository.com/artifact/org.slf4j/slf4j-api -->
    <dependency>
      <groupId>org.slf4j</groupId>
      <artifactId>slf4j-api</artifactId>
      <version>1.7.25</version>
    </dependency>
    <!-- https://mvnrepository.com/artifact/org.slf4j/slf4j-log4j12 -->
    <dependency>
      <groupId>org.slf4j</groupId>
      <artifactId>slf4j-log4j12</artifactId>
      <version>1.7.25</version>
    </dependency>
    <dependency>
      <groupId>org.apache.commons</groupId>
      <artifactId>commons-lang3</artifactId>
      <version>3.8.1</version>
    </dependency>

log4j配置

log4j.rootLogger = info,console

log4j.appender.console = org.apache.log4j.ConsoleAppender
log4j.appender.console.Target = System.out
log4j.appender.console.layout = org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern =  %p %d{yyyy-MM-dd HH:mm:ss} %c - %m%n

生产者

package com.baizhi.jsy.kafkaapi;
import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerConfig;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.apache.kafka.common.serialization.StringSerializer;
import java.text.DecimalFormat;
import java.util.Properties;
public class ProductKafka {
    public static void main(String[] args) {
        //创建生产者
        Properties properties = new Properties();
        properties.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG,"Centos:9092");
        properties.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
        properties.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG,StringSerializer.class.getName());

        //优化参数
        properties.put(ProducerConfig.BATCH_SIZE_CONFIG,1024*1024);//生产者尝试缓存记录，为每一个分区缓存一个mb的数据
        properties.put(ProducerConfig.LINGER_MS_CONFIG,500);//最多等待0.5秒.

        KafkaProducer<String, String> kafkaProducer = new KafkaProducer<String, String>(properties);

        for(int i=0;i<10;i++){
            DecimalFormat decimalFormat = new DecimalFormat("00");
            String format = decimalFormat.format(i);
            ProducerRecord<String, String> record = new ProducerRecord<>("topic01", "key" + format, "value" + format);
            kafkaProducer.send(record);
        }
        kafkaProducer.flush();
        kafkaProducer.close();
    }
}

消费者

package com.baizhi.jsy.kafkaapi;
import org.apache.kafka.clients.consumer.ConsumerConfig;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.apache.kafka.clients.consumer.ConsumerRecords;
import org.apache.kafka.clients.consumer.KafkaConsumer;
import org.apache.kafka.common.serialization.StringDeserializer;
import java.time.Duration;
import java.util.Arrays;
import java.util.Iterator;
import java.util.Properties;
import java.util.regex.Pattern;
public class ConsumerKafka {
    public static void main(String[] args) {
        //创建消费者
        Properties properties = new Properties();
        properties.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG,"Centos:9092");
        properties.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName());       properties.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG,StringDeserializer.class.getName());
        //第一次订阅topic的时候，读取的offect偏移量 latest/earliest
        properties.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG,"earliest");
        properties.put(ConsumerConfig.GROUP_ID_CONFIG,"group02");
        KafkaConsumer<String, String> kafkaConsumer = new KafkaConsumer<String, String>(properties);
        kafkaConsumer.subscribe(Arrays.asList("topic01"));
        try {
            while (true){
                //设置间隔多长时间取一次数据
                ConsumerRecords<String, String> consumerRecords = kafkaConsumer.poll(Duration.ofSeconds(1));
                //判断数据是否是空的
                if(!consumerRecords.isEmpty()){
                    Iterator<ConsumerRecord<String, String>> iterator = consumerRecords.iterator();
                    while (iterator.hasNext()){
                        ConsumerRecord<String, String> next = iterator.next();
                        String topic = next.topic();
                        System.out.println("topic = " + topic);
                        String key = next.key();
                        System.out.println("key = " + key);
                        String value = next.value();
                        System.out.println("value = " + value);
                        long offset = next.offset();
                        System.out.println("offset = " + offset);
                        int partition = next.partition();
                        System.out.println("partition = " + partition);
                        long timestamp = next.timestamp();
                        System.out.println("timestamp = " + timestamp);
                        System.out.println();
                    }
                }
            }
        } catch (Exception e) {
            e.printStackTrace();
        }finally {
            kafkaConsumer.close();
        }

    }
}

序列化的案例

创建user实体类

package com.baizhi.jsy.serializer;
import java.io.Serializable;
import java.util.Date;
public class User implements Serializable {
    private String id;
    private String name;
    private Date date;
    public User() {
    }
    public User(String id, String name, Date date) {
        this.id = id;
        this.name = name;
        this.date = date;
    }
    @Override
    public String toString() {
        return "User{" +
                "id='" + id + '\'' +
                ", name='" + name + '\'' +
                ", date=" + date +
                '}';
    }
    public String getId() {
        return id;
    }
    public void setId(String id) {
        this.id = id;
    }
    public String getName() {
        return name;
    }
    public void setName(String name) {
        this.name = name;
    }
    public Date getDate() {
        return date;
    }
    public void setDate(Date date) {
        this.date = date;
    }
}

user序列化

package com.baizhi.jsy.serializer;
import org.apache.commons.lang3.SerializationUtils;
import org.apache.kafka.common.serialization.Serializer;
import java.io.Serializable;
import java.util.Map;
public class Userserializer implements Serializer<Object> {
    @Override
    public void configure(Map<String, ?> map, boolean b){}
    @Override
    public byte[] serialize(String topic, Object data) {
        //导入commons-lang3包 用SerializationUtils直接序列化
        return SerializationUtils.serialize((Serializable) data);
    }
    @Override
    public void close(){}
}

user反序列化

package com.baizhi.jsy.serializer;
import org.apache.commons.lang3.SerializationUtils;
import org.apache.kafka.common.serialization.Deserializer;
import java.util.Map;
public class UserDeserializer implements Deserializer<Object>{
    @Override
    public void configure(Map<String, ?> map, boolean b) {}
    @Override
    public Object deserialize(String s, byte[] bytes) {
        return SerializationUtils.deserialize(bytes);
    }
    @Override
    public void close(){}
}

生产者

package com.baizhi.jsy.serializer;
import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerConfig;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.apache.kafka.common.serialization.StringSerializer;
import java.text.DecimalFormat;
import java.util.Date;
import java.util.Properties;
public class ProductKafkaUser {
    public static void main(String[] args) {
        //创建生产者
        Properties properties = new Properties();
        properties.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG,"Centos:9092");
        properties.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
        properties.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, "com.baizhi.jsy.serializer.Userserializer" );
        //优化参数
        properties.put(ProducerConfig.BATCH_SIZE_CONFIG,1024*1024);//生产者尝试缓存记录，为每一个分区缓存一个mb的数据
        properties.put(ProducerConfig.LINGER_MS_CONFIG,500);//最多等待0.5秒.
        KafkaProducer<String, User> kafkaProducer = new KafkaProducer<String, User>(properties);
        for(int i=0;i<10;i++){
            DecimalFormat decimalFormat = new DecimalFormat("00");
            String format = decimalFormat.format(i);
            ProducerRecord<String, User> record = new ProducerRecord<String, User>("topic02", "key" + format, new User(format,"User"+format,new Date()));
            kafkaProducer.send(record);
        }
        kafkaProducer.flush();
        kafkaProducer.close();
    }
}

消费者

//1.创建Kafka链接参数
Properties props=new Properties();
props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG,"CentOSA:9092,CentOSB:9092,CentOSC:9092");
props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName());
props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG,ObjectDeserializer.class.getName());
props.put(ConsumerConfig.GROUP_ID_CONFIG,"group01");

//2.创建Topic消费者
KafkaConsumer<String,User> consumer=new KafkaConsumer<String, User>(props);
//3.订阅topic开头的消息队列
consumer.subscribe(Pattern.compile("^topic.*$"));

while (true){
  ConsumerRecords<String, User> consumerRecords = consumer.poll(Duration.ofSeconds(1));
  Iterator<ConsumerRecord<String, User>> recordIterator = consumerRecords.iterator();
  while (recordIterator.hasNext()){
    ConsumerRecord<String, User> record = recordIterator.next();
    String key = record.key();
    User value = record.value();
    long offset = record.offset();
    int partition = record.partition();
    System.out.println("key:"+key+",value:"+value+",partition:"+partition+",offset:"+offset);
  }
}

自定义分区

源代码

/*
 * Licensed to the Apache Software Foundation (ASF) under one or more
 * contributor license agreements. See the NOTICE file distributed with
 * this work for additional information regarding copyright ownership.
 * The ASF licenses this file to You under the Apache License, Version 2.0
 * (the "License"); you may not use this file except in compliance with
 * the License. You may obtain a copy of the License at
 *
 *    http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package org.apache.kafka.clients.producer.internals;

import java.util.List;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentMap;
import java.util.concurrent.ThreadLocalRandom;
import java.util.concurrent.atomic.AtomicInteger;

import org.apache.kafka.clients.producer.Partitioner;
import org.apache.kafka.common.Cluster;
import org.apache.kafka.common.PartitionInfo;
import org.apache.kafka.common.utils.Utils;

/**
 * The default partitioning strategy:
 * <ul>
 * <li>If a partition is specified in the record, use it
 * <li>If no partition is specified but a key is present choose a partition based on a hash of the key
 * <li>If no partition or key is present choose a partition in a round-robin fashion
 */
public class DefaultPartitioner implements Partitioner {

    private final ConcurrentMap<String, AtomicInteger> topicCounterMap = new ConcurrentHashMap<>();

    public void configure(Map<String, ?> configs) {}

    /**
     * Compute the partition for the given record.
     *
     * @param topic The topic name
     * @param key The key to partition on (or null if no key)
     * @param keyBytes serialized key to partition on (or null if no key)
     * @param value The value to partition on or null
     * @param valueBytes serialized value to partition on or null
     * @param cluster The current cluster metadata
     */
    public int partition(String topic, Object key, byte[] keyBytes, Object value, byte[] valueBytes, Cluster cluster) {
        List<PartitionInfo> partitions = cluster.partitionsForTopic(topic);
        int numPartitions = partitions.size();
        if (keyBytes == null) {
            int nextValue = nextValue(topic);
            List<PartitionInfo> availablePartitions = cluster.availablePartitionsForTopic(topic);
            if (availablePartitions.size() > 0) {
                int part = Utils.toPositive(nextValue) % availablePartitions.size();
                return availablePartitions.get(part).partition();
            } else {
                // no partitions are available, give a non-available partition
                return Utils.toPositive(nextValue) % numPartitions;
            }
        } else {
            // hash the keyBytes to choose a partition
            return Utils.toPositive(Utils.murmur2(keyBytes)) % numPartitions;
        }
    }

    private int nextValue(String topic) {
        AtomicInteger counter = topicCounterMap.get(topic);
        if (null == counter) {
            counter = new AtomicInteger(ThreadLocalRandom.current().nextInt());
            AtomicInteger currentCounter = topicCounterMap.putIfAbsent(topic, counter);
            if (currentCounter != null) {
                counter = currentCounter;
            }
        }
        return counter.getAndIncrement();
    }

    public void close() {}

}

自定义分区

package com.baizhi.jsy.Partitions;
import org.apache.kafka.clients.producer.Partitioner;
import org.apache.kafka.common.Cluster;
import org.apache.kafka.common.PartitionInfo;
import org.apache.kafka.common.utils.Utils;

import java.util.List;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentMap;
import java.util.concurrent.ThreadLocalRandom;
import java.util.concurrent.atomic.AtomicInteger;
public class UserDefinePartitioner  implements Partitioner {
    private final ConcurrentMap<String, AtomicInteger> topicCounterMap = new ConcurrentHashMap<>();
    @Override
    public int partition(String topic, Object key, byte[] keyBytes, Object value, byte[] valueBytes, Cluster cluster) {
        List<PartitionInfo> partitionInfos = cluster.partitionsForTopic(topic);
        int numPartitions = partitionInfos.size();
        System.out.println("numPartitions = " + numPartitions);
        if(keyBytes==null || keyBytes.length==0){
            int nextValue = nextValue(topic);
            Integer lx=Utils.toPositive(nextValue)%numPartitions;
            System.out.println("lx = " + lx);
            return lx;
        } else {
            Integer p =Utils.toPositive(Utils.murmur2(keyBytes)) % numPartitions;
            System.out.println("p = " + p);
            return p;
        }
    }
    private int nextValue(String topic) {
        AtomicInteger counter = topicCounterMap.get(topic);
        if (null == counter) {
            counter = new AtomicInteger(ThreadLocalRandom.current().nextInt());
            AtomicInteger currentCounter = topicCounterMap.putIfAbsent(topic, counter);
            if (currentCounter != null) {
                counter = currentCounter;
            }
        }
        return counter.getAndIncrement();
    }
    @Override
    public void close() {
        System.out.println("close");
    }
    @Override
    public void configure(Map<String, ?> configs) {
        System.out.println("configure");
    }
}

生产者

package com.baizhi.jsy.Partitions;
import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerConfig;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.apache.kafka.common.serialization.StringSerializer;
import java.text.DecimalFormat;
import java.util.Date;
import java.util.Properties;
public class PartitionProductKafkaUser {
    public static void main(String[] args) {
        //创建生产者
        Properties properties = new Properties();
        properties.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG,"Centos:9092");
        properties.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
        properties.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, "com.baizhi.jsy.serializer.Userserializer" );
        properties.put(ProducerConfig.PARTITIONER_CLASS_CONFIG,"com.baizhi.jsy.Partitions.UserDefinePartitioner");
        //优化参数
        properties.put(ProducerConfig.BATCH_SIZE_CONFIG,1024*1024);//生产者尝试缓存记录，为每一个分区缓存一个mb的数据
        properties.put(ProducerConfig.LINGER_MS_CONFIG,500);//最多等待0.5秒.
        KafkaProducer<String, User> kafkaProducer = new KafkaProducer<String, User>(properties);
        for(int i=0;i<10;i++){
            DecimalFormat decimalFormat = new DecimalFormat("00");
            String format = decimalFormat.format(i);
            //ProducerRecord<String, User> record = new ProducerRecord<String, User>("topic02", "key" + format, new User(format,"User"+format,new Date()));
            ProducerRecord<String, User> record = new ProducerRecord<String, User>("topic02", new User(format,"User"+format,new Date()));
            kafkaProducer.send(record);
        }
        kafkaProducer.flush();
        kafkaProducer.close();
    }
}

消费者

package com.baizhi.jsy.Partitions;
import org.apache.kafka.clients.consumer.ConsumerConfig;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.apache.kafka.clients.consumer.ConsumerRecords;
import org.apache.kafka.clients.consumer.KafkaConsumer;
import org.apache.kafka.common.serialization.StringDeserializer;
import java.time.Duration;
import java.util.Arrays;
import java.util.Iterator;
import java.util.Properties;
public class PartitionConsumerKafkaUser {
    public static void main(String[] args) {
        //创建消费者
        Properties properties = new Properties();
        properties.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG,"Centos:9092");
        properties.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName());
        properties.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG,"com.baizhi.jsy.serializer.UserDeserializer");
        //第一次订阅读取的位置，默认是最新的latest
        //properties.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG,"earliest");
        properties.put(ConsumerConfig.GROUP_ID_CONFIG,"g2");
        KafkaConsumer<String, User> kafkaConsumer = new KafkaConsumer<>(properties);
        kafkaConsumer.subscribe(Arrays.asList("topic02"));
        try {
            while (true){
                //设置间隔多长时间取一次数据
                ConsumerRecords<String, User> consumerRecords = kafkaConsumer.poll(Duration.ofSeconds(1));
                //判断数据是否是空的
                if(!consumerRecords.isEmpty()){
                    Iterator<ConsumerRecord<String, User>> iterator = consumerRecords.iterator();
                    while (iterator.hasNext()){
                        ConsumerRecord<String, User> next = iterator.next();
                        String topic = next.topic();
                        String key = next.key();
                        User user = next.value();
                        long offset = next.offset();
                        int partition = next.partition();
                        long timestamp = next.timestamp();
                        System.out.println("topic = " + topic+"\t"+"key = " + key+"\t"+"User = " + user+"\t"+"offset = " + offset+"\t"+"partition = " + partition+"\t"+"timestamp = " + timestamp);
                        System.out.println();
                    }
                }
            }
        } catch (Exception e) {
            e.printStackTrace();
        }finally {
            kafkaConsumer.close();
        }

    }
}
姜同学的学习笔记
关注
  • 0
    点赞
  • 踩
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
kafka-spring-boot-example：Spring Boot应用程序显示与Kafka一起使用的基本配置
01-30
该示例应用程序演示如何使用基本的Spring Boot配置将生产者设置为具有多个分区的主题以及具有三个不同使用者的使用者组。 完整的解释在The Practical Developer网站上： 。 多种序列化/反序列化格式 为了说明不同的...
Insulator:客户端用户界面，可检查Kafka主题，消费，生产等等
04-06
Avro和String反序列化 根据记录时间戳寻求 轻松搜索和过滤 架构注册表 列出主题 显示给定主题的所有架构版本 制片人 具有基于模式的自动完成功能的Avro生产者 弦乐制作人 发送墓碑 :construction: 消费群体 :...
KafkaJavaApi操作案例：Kafka数据分区(一)
03-19 3718
案例一: 当没有指定分区号和key时使用轮训（轮训分区）策略存储数据 Producer生产者代码: public class Producer { //程序的入口 public static void main(String[] args){ //1、配置kafka集群 Properties props = new Properties(); ...
Kafka分区机制与代码示例
liuwei0376的专栏
03-27 1万+
Kafka中，topic是逻辑上的概念，而partition是物理上的概念。不用担心，这些对用户来说是透明的。生产者（producer）只关心自己将消息发布到哪个topic，而消费者（consumer）只关心自己订阅了哪个topic上的消息，至少topic上的消息分布在哪些partition节点上，它本身并不关心。 如果没有分区的概念，那么topic的消息集合将集中于某一台服务器上，单节点的存储性能
Kafka消费者Consumer常用配置
weixin_34293902的博客
04-27 553
bootstrap.servers：broker服务器集群列表，格式为 host1:port1, host2:port2key.deserializer：定义序列化的接口value.deserializer：实现序列化接口的类group.id：消费者分组Idconsumer.timeout.ms：消费者连接超时时间，默认为5000毫秒zookeeper.connect：Zookeeper服务器地址...
Kafka的分区策略
热门推荐
weixin_41928342的博客
07-02 1万+
    在kafka中，每个topic一般会有很多个Partitions。为了使我们能够及时的消费消息！  我们也可能启动多个Consumer去消费分区，而每个Consumer同样会启动一个或者是多个streams去分别消费Topic里面的数据。我们又知道，在Kafka中存在着Consumer Group的概念，也就是在group.id 一样的Consumer，这些Consumer属于同一个Con...
【大数据开发】Kafka——Kafka必知必会②
白色风车
10-22 210
一、封装工具类 下载consumer.properties、producer.properties文件到resources中 二、偏移量offset 三、自定义分区器
kafka 生产者
01-20
创建一个包含目标主题和内容的 ProducerRecord 对象，可指定键或分区，发送前要把对象序列化成字符数组。 数据被传给分区器，如果指定了分区就直接把指定的分区返回。如果没有指定分区，分区器就根据 ProducerRecord...
worldwindjava源码-franzy:ClojureKafka客户端，支持Kafka生产者、消费者、重新平衡、管理和验证
06-05
消费者、生产者、分区器、回调、序列化器和反序列化器的库。 此外，还有用于管理、测试、模拟、运行嵌入式 Kafka 代理和 Zookeeper 集群等的库。 Franzy 的主要目标是让使用 Clojure 的 Kafka 工作更轻松。 Franzy ...
经典面试题-Kafka
01-07
Kafka 中的分区器、序列化器、拦截器是否了解？它们之间的处理顺序是什么？ Kafka 生产者客户端的整体结构是什么样子的？使用了几个线程来处理？分别是什么？ “消费组中的消费者个数如果超过 topic 的分区，那么就...
Kafka生产者和消费者以及自定义序列化和反序列化
m0_37135879的博客
11-21 2470
一.生产者消费者 1.生产者生产消息方式 同步方式 public class Myproducer1 { public static void main(String[] args) { Map<String, Object> configs = new HashMap<>(); configs.put("bootstrap.servers", "Linux122:9092"); configs.put("key.seriali
kafka 自定义分区实例
liuxiangke0210的专栏
05-08 4539
第一步：使用./kafka-topics.sh 命令创建topic及partitions 分区数 bin/kafka-topics.sh --create --zookeeper 192.168.31.130:2181 --replication-factor 2 --partitions 3 --topic Topic-test 第二步：实现org.apache.kafka.cli
Kafka分区机制介绍与示例
liuxiangke0210的专栏
05-08 8230
原文链接：http://lxw1234.com/archives/2015/10/538.htm Kafka中可以将Topic从物理上划分成一个或多个分区（Partition），每个分区在物理上对应一个文件夹，以”topicName_partitionIndex”的命名方式命名，该文件夹下存储这个分区的所有消息(.log)和索引文件(.index)，这使得Kafka的吞吐率可以水
Kafka学习（9）- kafka使用Java开发-5-自定义分区实例
yiguang_820的博客
01-12 1143
目录： 操作流程： 创建一个maven项目，项目结构目录 生产者代码（往主题test-1，key为139开头的号码（此号码属于移动）生产消息，并在参数配置中使用自定义分区） 消费者代码（消费主题test-1的消息） 自定义分区类（作用：根据手机号码类型分区：联通号的消息放到分区0，移动号的消息放到分区1，电信号的消息放到分区2） 开启zookeeper和kafka 使用ka...
SpringBoot集成kafka(生产者和消费者)，自定义kafka的value序列化方式（附完整demo）
supreme000的博客
07-30 5317
Windows下kafka的安装参考文章：https://blog.csdn.net/github_38482082/article/details/82112641 maven依赖： <dependencies> <dependency> <groupId>org.springframework.bo...
Kafka： ------ 事务控制、消费者隔离级别read_committed与read_uncommitted区别
weixin_45764675的博客
04-14 4481
导入依赖 <dependency> <groupId>org.apache.kafka</groupId> <artifactId>kafka-clients</artifactId> <version>2.2.0</version> </dependency&...
Kafka： ------ Spring Boot整合Kafka 、接受数据、发送数据
weixin_45764675的博客
04-14 2728
- 引入依赖 <parent> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-parent</artifactId> <version>2.1.5.RELEASE</version> </pa...
kafka： ------ 数据同步机制
weixin_45764675的博客
04-15 2547
Kafka API实战 依赖 <!-- https://mvnrepository.com/artifact/org.apache.kafka/kafka-clients --> <dependency> <groupId>org.apache.kafka</groupId> <artifactId>kafka-clients&l...
Kafka： ------ 开启幂等性精准一次写入
weixin_45764675的博客
04-14 1711
幂等性 HTTP/1.1中对幂等性的定义是：一次和多次请求某一个资源对于资源本身应该具有同样的结果（网络超时等问题除外）。也就是说，其任意多次执行对资源本身所产生的影响均与一次执行的影响相同。 Methods can also have the property of “idempotence” in that (aside from error or expiration issues) th...
kafka-client0.10.0.1.jar
最新发布
08-20
它提供了高级的Producer API和Consumer API，使开发人员能够创建生产者和消费者，并使用不同的主题和分区发送和接收消息。 此外，kafka-client0.10.0.1.jar还提供了一些其他功能，例如事务支持和消息压缩。它还支持...

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值