SpringBoot批量获取Kafka消息（二）

导读：通过本文可以学习到SpringBoot整合Kafka、逐条消费Kafka消息、批量收取Kafka消息以及SpringBoot整合EmbeddedKafka进行无需启动外部kafka-server的单元测试。

 

我们似乎经常遇到一个问题，测试kafka相关的一些功能的时候，总需要启动外部的kafka-server服务器。但如果kafka-server和你开发机器不是同一个网络或者有网络隔离策略的时候，这时候你只能在本地启动你的zookeeper和kafka-server，非常麻烦。

接着上文，本文将介绍spring-kafka-test无需启动外部kafka-server，也能测试Kafka相关功能，例如测试@KafkaListener。

 

一、稍作修改

1、增加spring-kafka-test依赖。

<dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <!-- https://mvnrepository.com/artifact/org.springframework.kafka/spring-kafka -->
        <dependency>
            <groupId>org.springframework.kafka</groupId>
            <artifactId>spring-kafka</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
            <exclusions>
                <exclusion>
                    <groupId>org.junit.vintage</groupId>
                    <artifactId>junit-vintage-engine</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <version>1.18.12</version>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.kafka</groupId>
            <artifactId>spring-kafka-test</artifactId>
            <scope>test</scope>
        </dependency>

    </dependencies>

2、避免干扰，换个topic。

public interface Const {

    String KAFKA_BATCH_DEMO_TOPIC = "kafka-batch-demo-topic1";
}

3、KafkaBatchReceiver.java 保持不变。

@Slf4j
@Component
public class KafkaBatchReceiver {

    @KafkaListener(id = "kafka-batch-demo", topics = Const.KAFKA_BATCH_DEMO_TOPIC)
    public void receive(List<ConsumerRecord<String, String>> records) {
        for (ConsumerRecord<String, String> record : records) {
            receiveMesage(record);
        }
    }
    private void receiveMesage(ConsumerRecord<String, String> record) {

        if (null == record || StringUtils.isEmpty(record.value())) {

            log.warn("BatchKafkaReceiver record is null or record.value is empty.");
            return;
        }

        String reqJson = record.value();
        log.info("BatchKafkaReceiver {}", reqJson);


    }
}

4、增加测试类EmbeddedKafkaTest.java

/**
 * @author 
 * @date 2020/5/7 5:42 下午
 */
@RunWith(SpringRunner.class)
@SpringBootTest
@FixMethodOrder(MethodSorters.NAME_ASCENDING)
@Slf4j
@DirtiesContext
@EmbeddedKafka(topics = { Const.KAFKA_BATCH_DEMO_TOPIC })
@TestPropertySource(properties = { "spring.kafka.bootstrap-servers=${spring.embedded.kafka.brokers}" })
public class EmbeddedKafkaTest {

    @Autowired
    private EmbeddedKafkaBroker embeddedKafkaBroker;

    @Test
    public void testSend() throws IOException {

        Map<String, Object> configs = new HashMap<>(KafkaTestUtils.producerProps(embeddedKafkaBroker));
        Producer<String, String> producer = new DefaultKafkaProducerFactory<>(configs, new StringSerializer(), new StringSerializer()).createProducer();

        for (int i = 0; i < 10; i++) {

            producer.send(new ProducerRecord<>(Const.KAFKA_BATCH_DEMO_TOPIC, "my-aggregate-id", "my-test-value2"));
            producer.flush();

        }

        System.in.read();
    }


}

5、运行单元测试

2020-05-07 21:27:29.353 [kafka-batch-demo-0-C-1] INFO  o.s.kafka.listener.KafkaMessageListenerContainer - kafka-batch-demo: partitions assigned: [kafka-batch-demo-topic1-0, kafka-batch-demo-topic1-1]
2020-05-07 21:27:29.391 [kafka-batch-demo-0-C-1] INFO  com.mmc.kafka.demo.kafka.KafkaBatchReceiver - BatchKafkaReceiver my-test-value2
2020-05-07 21:27:29.391 [kafka-batch-demo-0-C-1] INFO  com.mmc.kafka.demo.kafka.KafkaBatchReceiver - BatchKafkaReceiver my-test-value2
2020-05-07 21:27:29.391 [kafka-batch-demo-0-C-1] INFO  com.mmc.kafka.demo.kafka.KafkaBatchReceiver - BatchKafkaReceiver my-test-value2
2020-05-07 21:27:29.391 [kafka-batch-demo-0-C-1] INFO  com.mmc.kafka.demo.kafka.KafkaBatchReceiver - BatchKafkaReceiver my-test-value2
2020-05-07 21:27:29.391 [kafka-batch-demo-0-C-1] INFO  com.mmc.kafka.demo.kafka.KafkaBatchReceiver - BatchKafkaReceiver my-test-value2
2020-05-07 21:27:29.391 [kafka-batch-demo-0-C-1] INFO  com.mmc.kafka.demo.kafka.KafkaBatchReceiver - BatchKafkaReceiver my-test-value2
2020-05-07 21:27:29.391 [kafka-batch-demo-0-C-1] INFO  com.mmc.kafka.demo.kafka.KafkaBatchReceiver - BatchKafkaReceiver my-test-value2
2020-05-07 21:27:29.391 [kafka-batch-demo-0-C-1] INFO  com.mmc.kafka.demo.kafka.KafkaBatchReceiver - BatchKafkaReceiver my-test-value2
2020-05-07 21:27:29.391 [kafka-batch-demo-0-C-1] INFO  com.mmc.kafka.demo.kafka.KafkaBatchReceiver - BatchKafkaReceiver my-test-value2
2020-05-07 21:27:29.391 [kafka-batch-demo-0-C-1] INFO  com.mmc.kafka.demo.kafka.KafkaBatchReceiver - BatchKafkaReceiver my-test-value2

6、完整项目结构

二、关于内推

你是否苦恼于自己技术不差但总是得不到大厂面试机会，社招流程慢进度不透明，面试发挥不好等等问题，关注一下公众号即可获得各种大厂内推资格，各种优质岗位等着你，欢迎投递。