背景
多个租户的消息推送到kafka,同一个消费者组的多个消费者消费。导致同一时间多个消费者消费到同一个租户的信息,造成数据库行级锁等待。
解决方案
把同一个租户的消息发送到kafka时,取模路由到同一个分区,这样同一个租户的消息只会被一个消费者消费,避免数据库多个事务中对同一个租户操作造成行级锁等待。
spring.cloud.stream.bindings.billChargeRecordVoiceRobotOutput.producer.partitionKeyExtractorName
=billVoiceRobotPartitionKeyExtractor
spring.cloud.stream.bindings.billChargeRecordVoiceRobotOutput.producer.partitionSelectorName
=billVoiceRobotPartitionSelector
// 提取message的key
public class BillVoiceRobotPartitionKeyExtractor implements PartitionKeyExtractorStrategy {
private static final Logger logger = LoggerFactory.getLogger(BillVoiceRobotPartitionKeyExtractor.class);
@Override
public Object extractKey(Message<?> message) {
Object payload = message.getPayload();
if (payload instanceof String) {
MealResourceVoiceRobotEventDto eventDto = JSON.parseObject(payload.toString(), MealResourceVoiceRobotEventDto.class);
if (StringUtils.isBlank(eventDto.getTenantId()) || null == eventDto.getCallType()) {
return null;
}
String str = eventDto.getTenantId() + eventDto.getCallType().toString();
return str.hashCode();
}
return null;
}
}
//根据key,选择路由到哪个partition
public class BillVoiceRobotPartitionSelectorClass implements PartitionSelectorStrategy {
private static final Logger logger = LoggerFactory.getLogger(BillVoiceRobotPartitionSelectorClass.class);
@Override
public int selectPartition(Object key, int partitionCount) {
//logger.info("消息载荷的key:{},partitionCount:{}", key, partitionCount);
return key == null ? 0 : ((int)key % partitionCount);
}
}
原理
kafka一个分区的数据只会被同一个消费者组的某一个消费者消费。