mybatis-plus 雪花算法id冲突问题解决、雪花算法id冲突、雪花算法、id冲突解决、id、id冲突、主键冲突

梦想2018

于 2024-09-05 10:02:40 发布

阅读量320

点赞数 5

文章标签： mybatis 算法雪花算法

本文链接：https://blog.csdn.net/ab411919134/article/details/141923295

版权

mybatis-plus 雪花算法id冲突问题解决、雪花算法id冲突、雪花算法、id冲突解决、id、id冲突、主键冲突

问题：
k8s里起了多个pod，发现mybatis-plus的雪花算法不同pod之前生成了相同的id

问题原因：
mybatis-plus默认id生成器生成datacenterId时是读取的机器网卡mac地址后两个字节，生成一个0~31的数字，这里有很大机率生成相同的datacenterId值
mybatis-plus默认id生成器生成workerId时是读取的jvm pid，但是k8s里pod的jvm pid不作处理时默认都是1，导致datacenterId + workerId有很大机率不同的pod拥有相同的值

分析日志：
ee:dc:39:3f:1a:53
531a >> 6 = 0101001100
0101001100 % 32 = 01100

86:65:49:0a:d1:f4
f4d1 >> 6 = 1111010011
1111010011 % 32 = 10011

e2:71:3a:cc:31:c3
c331 >> 6 = 1100001100
1100001100 % 32 = 01100

解决思路：
不同的pod之间分配唯一的datacenterId、workerId，使生成的雪花算法id不会产生冲突

解决方案（已实际使用并已通过测试）：

//程序配置
application.yml:

mybatis-plus-ext:
  snowflake:
    redis-key: smarthome.mybatis-plus-ext.snowflake.redis-key #mybatis-plus 雪花算法生成器配置。解决多个pod情况下生成的 数据中心id+workerId 重复的问题的redis键值

//配置类
MybatisPlusSnowflakeConfig.java:

package com.onbright.smarthome.config;

import com.baomidou.mybatisplus.core.incrementer.DefaultIdentifierGenerator;
import com.baomidou.mybatisplus.core.incrementer.IdentifierGenerator;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.tuple.MutablePair;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.redis.core.RedisTemplate;

/**
 * <p>
 * mybatis-plus 雪花算法生成器配置。解决多个pod情况下生成的 数据中心id+workerId 重复的问题
 * </p>
 */
@Slf4j
@Configuration
public class MybatisPlusSnowflakeConfig {

    @Autowired
    private RedisTemplate redisTemplate;
    @Value("${mybatis-plus-ext.snowflake.redis-key}")
    private String redisKey;

    @Bean
    public IdentifierGenerator identifierGenerator(){
        //tested
        Long redisIncrementValue = redisTemplate.opsForValue().increment(redisKey, 1L);
        if (redisIncrementValue == null || redisIncrementValue < 0L){
            throw new RuntimeException(String.format("程序启动失败, 使用redis获取到的雪花算法索引值异常, %s", redisIncrementValue));
        }

        //tested
        //从redis索引值中取出dataCenterId、workerId
        MutablePair<Long, Long> longLongMutablePair = generateDatacenterIdAndWorkerIdFromIndex(redisIncrementValue);
        Long dataCenterId = longLongMutablePair.getLeft();
        Long workerId = longLongMutablePair.getRight();

        //tested
        //再次检查dataCenterId、workerId值
        if (dataCenterId == null || dataCenterId < 0L || dataCenterId > 31L
                || workerId == null || workerId < 0L || workerId > 31L
        ){
            throw new RuntimeException(String.format("程序启动失败, 使用雪花算法索引值算出的dataCenterId,workerId异常, %s, %s, %s", redisIncrementValue, dataCenterId, workerId));
        }

        log.info("mybatis-plus雪花算法datacenterId,workerId生成, redisIncrementValue: {}, dataCenterId: {}, workerId: {}", redisIncrementValue, dataCenterId, workerId);

        return new DefaultIdentifierGenerator(workerId, dataCenterId);
    }

    /**
     * 从index值中生成雪花算法的datacenterId(5bit)、workerId(5bit)
     *
     * @param index 索引值，如：1。必须为大于等于0的值
     * @return 返回生成好的用MutablePair包装的datacenterId(left)、workerId(right)，其中datacenterId值范围为0~31，workerId值范围也为0~31，如：{0,0}、{1、1}、{0,1}、{0,31}、{31,31}
     * @throws IllegalArgumentException 当参数索引值小于0时会抛出异常
     */
    public static MutablePair<Long, Long> generateDatacenterIdAndWorkerIdFromIndex(long index) {
        if (index < 0) {
            throw new IllegalArgumentException("索引值不能为负数");
        }

        long indexBucket = index % 1024L;
        long dataCenterId = indexBucket / 32L;
        long workerId = indexBucket % 32L;

        return MutablePair.of(dataCenterId, workerId);
    }

}


//测试核心函数
MybatisPlusSnowflakeConfigTest.java:

package com.onbright.smarthome.config;

import org.apache.commons.lang3.tuple.MutablePair;
import org.junit.Assert;
import org.junit.Test;


public class MybatisPlusSnowflakeConfigTest {

    @Test
    public void generateDatacenterIdAndWorkerIdFromIndex() {
        //legal
        {
            MutablePair<Long, Long> longLongMutablePair = MybatisPlusSnowflakeConfig.generateDatacenterIdAndWorkerIdFromIndex(0L);
            Assert.assertEquals(0L, longLongMutablePair.getLeft().longValue());
            Assert.assertEquals(0L, longLongMutablePair.getRight().longValue());
        }
        {
            MutablePair<Long, Long> longLongMutablePair = MybatisPlusSnowflakeConfig.generateDatacenterIdAndWorkerIdFromIndex(1L);
            Assert.assertEquals(0L, longLongMutablePair.getLeft().longValue());
            Assert.assertEquals(1L, longLongMutablePair.getRight().longValue());
        }
        {
            MutablePair<Long, Long> longLongMutablePair = MybatisPlusSnowflakeConfig.generateDatacenterIdAndWorkerIdFromIndex(2L);
            Assert.assertEquals(0L, longLongMutablePair.getLeft().longValue());
            Assert.assertEquals(2L, longLongMutablePair.getRight().longValue());
        }
        {
            MutablePair<Long, Long> longLongMutablePair = MybatisPlusSnowflakeConfig.generateDatacenterIdAndWorkerIdFromIndex(31L);
            Assert.assertEquals(0L, longLongMutablePair.getLeft().longValue());
            Assert.assertEquals(31L, longLongMutablePair.getRight().longValue());
        }
        {
            MutablePair<Long, Long> longLongMutablePair = MybatisPlusSnowflakeConfig.generateDatacenterIdAndWorkerIdFromIndex(32L);
            Assert.assertEquals(1L, longLongMutablePair.getLeft().longValue());
            Assert.assertEquals(0L, longLongMutablePair.getRight().longValue());
        }
        {
            MutablePair<Long, Long> longLongMutablePair = MybatisPlusSnowflakeConfig.generateDatacenterIdAndWorkerIdFromIndex(33L);
            Assert.assertEquals(1L, longLongMutablePair.getLeft().longValue());
            Assert.assertEquals(1L, longLongMutablePair.getRight().longValue());
        }
        {
            MutablePair<Long, Long> longLongMutablePair = MybatisPlusSnowflakeConfig.generateDatacenterIdAndWorkerIdFromIndex(63L);
            Assert.assertEquals(1L, longLongMutablePair.getLeft().longValue());
            Assert.assertEquals(31L, longLongMutablePair.getRight().longValue());
        }
        {
            MutablePair<Long, Long> longLongMutablePair = MybatisPlusSnowflakeConfig.generateDatacenterIdAndWorkerIdFromIndex(64L);
            Assert.assertEquals(2L, longLongMutablePair.getLeft().longValue());
            Assert.assertEquals(0L, longLongMutablePair.getRight().longValue());
        }
        {
            MutablePair<Long, Long> longLongMutablePair = MybatisPlusSnowflakeConfig.generateDatacenterIdAndWorkerIdFromIndex(991L);
            Assert.assertEquals(30L, longLongMutablePair.getLeft().longValue());
            Assert.assertEquals(31L, longLongMutablePair.getRight().longValue());
        }
        {
            MutablePair<Long, Long> longLongMutablePair = MybatisPlusSnowflakeConfig.generateDatacenterIdAndWorkerIdFromIndex(992L);
            Assert.assertEquals(31L, longLongMutablePair.getLeft().longValue());
            Assert.assertEquals(0L, longLongMutablePair.getRight().longValue());
        }
        {
            MutablePair<Long, Long> longLongMutablePair = MybatisPlusSnowflakeConfig.generateDatacenterIdAndWorkerIdFromIndex(993L);
            Assert.assertEquals(31L, longLongMutablePair.getLeft().longValue());
            Assert.assertEquals(1L, longLongMutablePair.getRight().longValue());
        }
        {
            MutablePair<Long, Long> longLongMutablePair = MybatisPlusSnowflakeConfig.generateDatacenterIdAndWorkerIdFromIndex(1023L);
            Assert.assertEquals(31L, longLongMutablePair.getLeft().longValue());
            Assert.assertEquals(31L, longLongMutablePair.getRight().longValue());
        }
        {
            MutablePair<Long, Long> longLongMutablePair = MybatisPlusSnowflakeConfig.generateDatacenterIdAndWorkerIdFromIndex(1024L);
            Assert.assertEquals(0L, longLongMutablePair.getLeft().longValue());
            Assert.assertEquals(0L, longLongMutablePair.getRight().longValue());
        }

        //integer.max
        {
            MutablePair<Long, Long> longLongMutablePair = MybatisPlusSnowflakeConfig.generateDatacenterIdAndWorkerIdFromIndex(2147483647L);
            Assert.assertEquals(31L, longLongMutablePair.getLeft().longValue());
            Assert.assertEquals(31L, longLongMutablePair.getRight().longValue());
        }
        //long.max
        {
            MutablePair<Long, Long> longLongMutablePair = MybatisPlusSnowflakeConfig.generateDatacenterIdAndWorkerIdFromIndex(9223372036854775807L);
            Assert.assertEquals(31L, longLongMutablePair.getLeft().longValue());
            Assert.assertEquals(31L, longLongMutablePair.getRight().longValue());
        }

        //illegal
        {
            Assert.assertEquals("索引值不能为负数", Assert.assertThrows(IllegalArgumentException.class, () -> MybatisPlusSnowflakeConfig.generateDatacenterIdAndWorkerIdFromIndex(-1)).getMessage());
        }
    }
}