kafka自定义组件：自定义生产者(含自定义分区)，自定义消费者，自定义拦截器

• 自定义组件：自定义生产者(含自定义分区)，自定义消费者，自定义拦截器

• 自定义生产者：

• 官网地址：https://kafka.apache.org/26/javadoc/index.html?org/apache/kafka/clients/producer/KafkaProducer.html

   * 1. 创建生产者配置对象
   * 2. 添加配置信息
   * 3. 创建生产者对象
   * 4. 调用send发送消息
   * 5. 关闭资源
   * 6. 创建静态内部类，返回当前类对象
    //代码：
       public class KFKClient {
      public static void main(String[] args) throws IOException, InterruptedException {
          Properties properties = new Properties();
          properties.load(KFKClient.class.getClassLoader().getResourceAsStream("kfk.properties"));
          properties.put(ProducerConfig.INTERCEPTOR_CLASSES_CONFIG,"com.Raise.kafka.Interceptor.MyInterceptor");
          KafkaProducer producer = new KafkaProducer<>(properties);
          for (int i = 0; i < 10; i++) {
              producer.send(new ProducerRecord("first2", "hello world" + i), new Callback() {
                  @Override
                  public void onCompletion(RecordMetadata recordMetadata, Exception e) {
                      int partition = recordMetadata.partition();
                      System.out.println(recordMetadata.toString());
                  }
              });
          }
          producer.close();
      }
  }
  //配置文件：
  #key=value
  bootstrap.servers=localhost:9092
  #设置ack
  acks=all
  #重试次数
  retries=1
  #批次大小 默认16K
  batch.size=16384
  #等待时间
  linger.ms=1
  # RecordAccumulator缓冲区大小 默认32M
  buffer.memory=33554432
  #key,value序列化
  key.serializer=org.apache.kafka.common.serialization.StringSerializer
  value.serializer=org.apache.kafka.common.serialization.StringSerializer
  #自定义分区
  PARTITIONER_CLASS_CONFIG=com.Raise.kafka.partition.MyPartition
  

• 自定义消费者

• 官网案例：https://kafka.apache.org/22/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html

   * 1. 创建消费者配置类
   * 2. 添加配置
   * 3. 创建消费者对象
   * 4. 设置消费的主题
   * 5. 挂起消费数据
   //程序
       public class MyConsumer {
      public static void main(String[] args) throws IOException {
          Properties properties = new Properties();
          // 2. 添加配置参数
          // 添加连接
          properties.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
          // 配置序列化 必须
          properties.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.StringDeserializer");
          properties.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.StringDeserializer");
          // 配置消费者组
          properties.put(ConsumerConfig.GROUP_ID_CONFIG, "test");
          // 是否自动提交offset
          properties.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, "true");
          // 提交offset的时间周期
          properties.put(ConsumerConfig.AUTO_COMMIT_INTERVAL_MS_CONFIG, "1000");
          KafkaConsumer<String, String> consumer = new KafkaConsumer<>(properties);
          ArrayList<String> list = new ArrayList<>();
          list.add("first2");
          consumer.subscribe(list);
          while (true){
              ConsumerRecords<String, String> consumerRecords = consumer.poll(Duration.ofSeconds(1));
              for (ConsumerRecord<String, String> consumerRecord : consumerRecords) {
                  System.out.println(consumerRecord.topic() + "->" + consumerRecord.value());
              }
          }
      }
  }
  

• 自定义拦截器

  对于producer而言，interceptor使得用户在消息发送前以及producer回调逻辑前有机会对消息做一些定制化需求，比如修改消息等。同时，producer允许用户指定多个interceptor按序作用于同一条消息从而形成一个拦截链(interceptor chain)。Intercetpor的实现接口是org.apache.kafka.clients.producer.ProducerInterceptor
      //程序：
      public class MyInterceptor implements ProducerInterceptor<String,String> {
  
      private int success;
      private int failed;
  
      @Override
      public ProducerRecord<String, String> onSend(ProducerRecord<String, String> producerRecord) {
          //拦截器
          String value = producerRecord.value();
     	    String value = System.currentTimeMillis() + " -> " + record.value();
          return new ProducerRecord<>(producerRecord.topic(),producerRecord.partition(),producerRecord.timestamp(),producerRecord.key(),value,producerRecord.headers());
      }
  
      @Override
      public void onAcknowledgement(RecordMetadata recordMetadata, Exception e) {
          if (e == null){
              success++;
          }else {
              failed++;
          }
      }
  
      @Override
      public void close() {
          System.out.println("success->" + success + "\tfailed->" + failed);
      }
  
      @Override
      public void configure(Map<String, ?> map) {
  
      }
  }
  // 不能与生产者的类写到一个主类里面，需要public修饰