Spring Cloud Stream消费失败后的处理策略（四）：使用DLQ队列（Kafka版）

Spring Cloud Stream消费失败后的处理策略（四）：使用DLQ队列（Kafka版）

之前写了一个一个DLQ的rabbit版本，接下来说一下kafka版本的。因为kafka的管理监控界面没有那么详细（我太菜，没找到）。

动手试一下：

准备一个会消费失败的例子，可以直接沿用前文的工程。也可以新建一个，然后创建如下代码的逻辑：

public class TestController {
    @Autowired
    private TestTopic testTopic;
    @RequestMapping("entity")
    public void TestEntity(@RequestParam String message){
        Message message1 = new Message("1", message, new Date()); testTopic.output1().send(MessageBuilder.withPayload(message1).build());
    }
}

@Data
@NoArgsConstructor
@AllArgsConstructor
public class Message implements Serializable {
    String id;
    String msg;
    Date date;
}

public interface TestTopic {
    String OUTPUT1 = "example-topic-output1";
    String INPUT1 = "example-topic-input1";
    String ERRORINPUT = "errorChannelInput";
    String ERROROUTPUT = "errorChannelOutput";

    @Output(OUTPUT1)
    MessageChannel output1();

    @Input(INPUT1)
    SubscribableChannel input1();

    @Input(ERRORINPUT)
    SubscribableChannel errorInput();

    @Output(ERROROUTPUT)
    MessageChannel errorOutput();
}

@Slf4j
@Component
@EnableBinding(TestTopic.class)
public class TestListener {
    @StreamListener(TestTopic.INPUT1)
    public void receiveEntity(Message message) {
        log.info("Received payload : {}", message);
        throw new RuntimeException("Message consumer failed!");
    }
}

消息消费的时候主动抛出了一个异常来模拟消息的消费失败。

在启动应用之前，还要记得配置一下输入输出通道对应的物理目标（topic名）、并设置一下分组，比如：

# 为消费者设置组，防止重复消费
spring.cloud.stream.bindings.example-topic-input1.group=Service-A
spring.cloud.stream.bindings.errorChannel.group=Service-A
# 为example-topic-input通道绑定topic
spring.cloud.stream.bindings.example-topic-input1.destination=greeting1
spring.cloud.stream.bindings.errorChannelInput.destination=errorTopic
# example-topic-input通道发送消息失败时的重试次数
spring.cloud.stream.bindings.example-topic-input1.consumer.max-attempts=1
# 输出example-topic-output通道绑定的topic
spring.cloud.stream.bindings.example-topic-output1.destination=greeting1
spring.cloud.stream.bindings.errorChannelOutput.destination=errorTopic
# kafka服务器地址
spring.cloud.stream.kafka.binder.brokers=ip:port
server.port=8080
# 是否绑定死信队列,会立即转发至死信队列，若失败则会发送3次
spring.cloud.stream.kafka.bindings.example-topic-input1.consumer.enable-dlq=true
spring.cloud.stream.kafka.bindings.example-topic-input1.consumer.dlq-name=errorTopic

这里加入了两个重要配置

spring.cloud.stream.kafka.bindings.example-topic-input1.consumer.enable-dlq=true，

spring.cloud.stream.kafka.bindings.example-topic-input1.consumer.dlq-name=errorTopic

用来开启DLQ（死信队列）并指明死信队列名。完成了上面配置之后，启动应用并访问`localhost:8080/entity?message=aaa

最后打印日志：

2019-09-18 16:37:13.129  INFO 9184 --- [container-0-C-1] c.c.f.stream.listener.TestListener       : Received payload : Message(id=1, msg=aaaAASAsa, date=Wed Sep 18 16:37:13 CST 2019)

org.springframework.messaging.MessagingException: Exception thrown while invoking com.chinatower.framework.stream.listener.TestListener#receiveEntity[1 args]; nested exception is java.lang.RuntimeException: Message consumer failed!
	at org.springframework.cloud.stream.binding.StreamListenerMessageHandler.handleRequestMessage(StreamListenerMessageHandler.java:63) ~[spring-cloud-stream-2.0.1.RELEASE.jar:2.0.1.RELEASE]

Caused by: java.lang.RuntimeException: Message consumer failed!
	at com.chinatower.framework.stream.listener.TestListener.receiveEntity(TestListener.java:26) ~[classes/:na]


2019-09-18 16:37:13.131 ERROR 9184 --- [container-0-C-1] o.s.integration.handler.LoggingHandler   : org.springframework.messaging.MessagingException: Exception thrown while invoking com.chinatower.framework.stream.listener.TestListener#receiveEntity[1 args]; nested exception is java.lang.RuntimeException: Message consumer failed!, failedMessage=GenericMessage [payload=byte[66], headers={kafka_offset=39, scst_nativeHeadersPresent=true, kafka_consumer=org.apache.kafka.clients.consumer.KafkaConsumer@3a3d1789, kafka_timestampType=CREATE_TIME, kafka_receivedMessageKey=null, kafka_receivedPartitionId=0, contentType=application/json, kafka_receivedTopic=greeting1, kafka_receivedTimestamp=1568795833086}]
	at org.springframework.cloud.stream.binding.StreamListenerMessageHandler.handleRequestMessage(StreamListenerMessageHandler.java:63)

Caused by: java.lang.RuntimeException: Message consumer failed!
	at com.chinatower.framework.stream.listener.TestListener.receiveEntity(TestListener.java:26)


2019-09-18 16:37:13.173  INFO 9184 --- [container-0-C-1] c.c.f.stream.listener.TestListener       : errorReceive payload : {"id":"1","msg":"aaaAASAsa","date":"2019-09-18T08:37:13.084+0000"}

如此一来，就可以做到自己接受死信队列里的消息，做预防措施。