spring-session之4 redis集群配置

spring-session之4 redis集群配置

前文,我们 spring-session之3 redis配置参数配置, 我们可以配置简单的ip和port,但是生产环境,我们的redis是做了集群,肯定不是单点,此时就不能单单hostName 和 port了, 怎么办?

好,今天的目标是:

  1. 配置spring-session redis 集群

1. 我们原来的 redis data 配置

1.1. spring-redis.xml

    <?xml version="1.0" encoding="UTF-8"?>
    <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xmlns:context="http://www.springframework.org/schema/context" xmlns:loxia="http://loxia.benjamin.cn/schema/core"
        xmlns:p="http://www.springframework.org/schema/p" xmlns:util="http://www.springframework.org/schema/util"
        xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
            http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd
            http://loxia.benjamin.cn/schema/core http://loxia.benjamin.cn/schema/core/loxia-spring-ext.xsd
            http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd   
            ">

        <bean id="redisconfig" class="redis.clients.jedis.JedisPoolConfig">
            <property  name="maxActive" >
                <value type="long">#{redis['redis.MaxActive']}</value>
            </property>
            <property  name="maxIdle" >
                <value type="long">#{redis['redis.MaxIdle']}</value>
            </property>
            <property  name="maxWait">
                <value type="long">#{redis['redis.MaxWait']}</value>
            </property>
            <property  name="testOnBorrow" >
                <value type="boolean">true</value>
            </property>
            <property  name="testOnReturn" >
                <value type="boolean">true</value>
            </property>
        </bean>

        <bean id="jedisSentinelPool" class="redis.clients.jedis.JedisSentinelPool">
            <constructor-arg ref="redisconfig"></constructor-arg>

            <constructor-arg type="java.util.Set">
                <set>
                    <value>#{redis['redis.sentinel1']}</value>
                    <value>#{redis['redis.sentinel2']}</value>
                    <value>#{redis['redis.sentinel3']}</value>
                </set>

            </constructor-arg>

            <constructor-arg >
                <value>#{redis['redis.mastername']}</value>
            </constructor-arg>

        </bean>
    </beans>

1.2 redis.properties

    #A string containing whitespace or comma separated host or IP addresses and port numbers of the form "host:port host2:port" or "host:port, host2:port".
    redis.sentinel1=redis01.public.test.baozun.cn:26379
    redis.sentinel2=redis02.public.test.baozun.cn:26379
    redis.sentinel3=redis03.public.test.baozun.cn:26379

    redis.mastername=testenv

    redis.timeout=1000
    redis.MaxActive=50
    redis.MaxIdle=60000
    redis.MaxWait=5000

    ##这个是我们自定义的 key前缀, 以便用同一套redis集群,适应多个不同的商城
    redis.keystart=feilongdev

2.redis 集群配置

那么我们怎么使用上面的环境来配置我们的spring-session呢?

使用通过上述的参数表格, JedisPoolConfig 可以使用,但是没有 JedisSentinelPool参数

而 JedisShardInfo属性并不是list,只是 包含了jedis服务器的一些信息 ,咋整?

这时候,我们来看看 JedisConnectionFactory 构造函数:

可以看出 ,构造函数支持 RedisSentinelConfiguration 或者 RedisClusterConfiguration

此处的 JedisConnectionFactory 是 spring-data-redis 内容, 有兴趣的可以看看, 这里也有入门版的教程,参见使用Spring Data Redis操作Redis(一)

2.1 RedisSentinelConfiguration

我们先使用 redis 哨兵机制的集群

那么我们尝试修改以下配置配置:

    <?xml version="1.0" encoding="UTF-8"?>
    <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context"
        xmlns:p="http://www.springframework.org/schema/p" xmlns:util="http://www.springframework.org/schema/util"
        xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
            http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd
            http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd   
            ">

        <context:annotation-config />

        <util:properties id="redis" location="classpath:redis.properties"></util:properties>

        <!-- RedisHttpSessionConfiguration -->
        <bean class="org.springframework.session.data.redis.config.annotation.web.http.RedisHttpSessionConfiguration" />

        <!--JedisConnectionFactory -->
        <bean class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory">
            <constructor-arg index="0">
                <bean class="org.springframework.data.redis.connection.RedisSentinelConfiguration">
                    <constructor-arg index="0" value="#{redis['redis.mastername']}" />
                    <constructor-arg index="1">
                        <set>
                            <value>#{redis['redis.sentinel1']}</value>
                            <value>#{redis['redis.sentinel2']}</value>
                            <value>#{redis['redis.sentinel3']}</value>
                        </set>
                    </constructor-arg>
                </bean>
            </constructor-arg>

            <constructor-arg index="1">
                <bean class="redis.clients.jedis.JedisPoolConfig">
                    <property name="maxActive" value="#{redis['redis.MaxActive']}" />
                    <property name="maxIdle" value="#{redis['redis.MaxIdle']}" />
                    <property name="maxWait" value="#{redis['redis.MaxWait']}" />
                    <property name="testOnBorrow" value="true" />
                    <property name="testOnReturn" value="true" />
                </bean>
            </constructor-arg>
        </bean>

    </beans>

启动应用

出现以下的错误

Caused by: org.springframework.beans.NotWritablePropertyException: Invalid property 'maxActive' of bean class [redis.clients.jedis.JedisPoolConfig]: Bean property 'maxActive' is not writable or has an invalid setter method. Does the parameter type of the setter match the return type of the getter?
    at org.springframework.beans.BeanWrapperImpl.setPropertyValue(BeanWrapperImpl.java:1044)
    at org.springframework.beans.BeanWrapperImpl.setPropertyValue(BeanWrapperImpl.java:904)
    at org.springframework.beans.AbstractPropertyAccessor.setPropertyValues(AbstractPropertyAccessor.java:75)
    at org.springframework.beans.AbstractPropertyAccessor.setPropertyValues(AbstractPropertyAccessor.java:57)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1452)
    ... 30 more

JedisPoolConfig 不支持 maxActive 属性了 see xetorthio/jedis#1031

好吧, 修改下配置 spring-session.xml

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context"
    xmlns:p="http://www.springframework.org/schema/p" xmlns:util="http://www.springframework.org/schema/util"
    xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
        http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd
        http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd   
        ">

    <context:annotation-config />

    <util:properties id="redis" location="classpath:redis.properties"></util:properties>

    <!-- RedisHttpSessionConfiguration -->
    <bean class="org.springframework.session.data.redis.config.annotation.web.http.RedisHttpSessionConfiguration" />

    <!--JedisConnectionFactory -->
    <bean class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory">
        <constructor-arg index="0">
            <bean class="org.springframework.data.redis.connection.RedisSentinelConfiguration">
                <constructor-arg index="0" value="#{redis['redis.mastername']}" />
                <constructor-arg index="1">
                    <set>
                        <value>#{redis['redis.sentinel1']}</value>
                        <value>#{redis['redis.sentinel2']}</value>
                        <value>#{redis['redis.sentinel3']}</value>
                    </set>
                </constructor-arg>
            </bean>
        </constructor-arg>

        <constructor-arg index="1">
            <bean class="redis.clients.jedis.JedisPoolConfig">
                <property name="maxIdle" value="#{redis['redis.MaxIdle']}" />
                <property name="testOnBorrow" value="true" />
                <property name="testOnReturn" value="true" />

                <!-- 新版jedis 不支持这个参数了 -->
                <!-- <property name="maxWait" value="#{redis['redis.MaxWait']}" /> -->
                <!-- <property name="maxActive" value="#{redis['redis.MaxActive']}" /> -->
            </bean>
        </constructor-arg>
    </bean>

</beans>

redis.properties

    #A string containing whitespace or comma separated host or IP addresses and port numbers of the form "host:port host2:port" or "host:port, host2:port".
    redis.sentinel1=redis01.public.test.baozun.cn:26379
    redis.sentinel2=redis02.public.test.baozun.cn:26379
    redis.sentinel3=redis03.public.test.baozun.cn:26379

    redis.mastername=testenv

    redis.timeout=1000

    redis.MaxIdle=60000

    #新版jedis 不支持这个参数了
    #redis.MaxActive=50
    #redis.MaxWait=5000

    redis.keystart=gowilddev

启动

报了新错

    Caused by: redis.clients.jedis.exceptions.JedisDataException: ERR Unsupported CONFIG parameter: notify-keyspace-events
        at redis.clients.jedis.Protocol.processError(Protocol.java:117)
        at redis.clients.jedis.Protocol.process(Protocol.java:151)
        at redis.clients.jedis.Protocol.read(Protocol.java:205)
        at redis.clients.jedis.Connection.readProtocolWithCheckingBroken(Connection.java:297)
        at redis.clients.jedis.Connection.getStatusCodeReply(Connection.java:196)
        at redis.clients.jedis.Jedis.configSet(Jedis.java:2612)
        at org.springframework.data.redis.connection.jedis.JedisConnection.setConfig(JedisConnection.java:633)
        ... 25 more

什么鬼,参见 ERR Unsupported CONFIG parameter: notify-keyspace-events

看来我是使用新的jedis 连接了老版本的redis集群

服务器上面的 redis 版本

找运维组重新搭建 redis新版本的集群环境

[vmuser@test01 ~]$ redis-cli --version
redis-cli 3.2.4

由于采用了新版的集群技术,所以我们只能使用 RedisClusterConfiguration

2.2 RedisClusterConfiguration

2.2.1 3主3从 Cluster服务端配置

我们先看看运维组兄弟配置的集群情况

[vmuser@test01 ~]$ /home/vmuser/redis-3.2.4/src/redis-trib.rb check 10.88.21.31:10000

可以看到是官网推荐的标准的 3主 3从 方案,参见 官方文档 http://redisdoc.com/topic/cluster-tutorial.html

master slave
10.88.21.31:10000 10.88.22.25:10001
10.88.22.25:10000 10.88.21.31:10002
10.88.21.31:10001 10.88.22.25:10002

2.2.2 spring配置

这时我们可以使用 org.springframework.data.redis.connection.RedisClusterConfiguration ,注意 since spring-data-redis 1.7

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context"
    xmlns:p="http://www.springframework.org/schema/p" xmlns:util="http://www.springframework.org/schema/util"
    xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
        http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd
        http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd   
        ">

    <context:annotation-config />

    <util:properties id="redis" location="classpath:redis-cluster.properties"></util:properties>

    <!-- RedisHttpSessionConfiguration -->
    <bean class="org.springframework.session.data.redis.config.annotation.web.http.RedisHttpSessionConfiguration" />

    <!--JedisConnectionFactory -->
    <bean class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory">
        <constructor-arg index="0">
            <bean class="org.springframework.data.redis.connection.RedisClusterConfiguration">
                <constructor-arg index="0">
                    <set>
                        <!-- slave -->
                        <!-- <value>10.88.22.25:10001</value>
                            <value>10.88.21.31:10002</value>
                            <value>10.88.22.25:10002</value> -->

                        <!-- master -->
                        <value>10.88.21.31:10000</value>
                        <value>10.88.22.25:10000</value>
                        <value>10.88.21.31:10001</value>
                    </set>
                </constructor-arg>

                <!-- 一般当此值设置过大时,容易报:Too many Cluster redirections -->
                <!-- <property name="maxRedirects">3</property> -->
            </bean>
        </constructor-arg>

        <constructor-arg index="1">
            <bean class="redis.clients.jedis.JedisPoolConfig">
                <property name="maxIdle" value="#{redis['redis.jedisPoolConfig.MaxIdle']}" />
                <property name="testOnBorrow" value="#{redis['redis.jedisPoolConfig.testOnBorrow']}" />
                <property name="testOnReturn" value="#{redis['redis.jedisPoolConfig.testOnReturn']}" />

                <!-- 新版jedis 不支持这个参数了 -->
                <!-- <property name="maxWait" value="#{redis['redis.jedisPoolConfig.MaxWait']}" /> -->
                <!-- <property name="maxActive" value="#{redis['redis.jedisPoolConfig.MaxActive']}" /> -->
            </bean>
        </constructor-arg>
    </bean>

</beans>

此时,我们仅需要配置 master 到 RedisClusterConfiguration 中,系统会自动分配

上面配置可以提取精炼到配置文件中

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context"
    xmlns:p="http://www.springframework.org/schema/p" xmlns:util="http://www.springframework.org/schema/util"
    xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
        http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd
        http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd   
        ">

    <context:annotation-config />

    <util:properties id="redis" location="classpath:redis-cluster.properties"></util:properties>

    <!-- RedisHttpSessionConfiguration -->
    <bean class="org.springframework.session.data.redis.config.annotation.web.http.RedisHttpSessionConfiguration" />

    <!--JedisConnectionFactory -->
    <bean class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory">
        <constructor-arg index="0">
            <!-- since spring-data-redis 1.7 -->
            <bean class="org.springframework.data.redis.connection.RedisClusterConfiguration">
                <constructor-arg index="0">
                    <set>
                        <value>#{redis['redis.redisClusterConfiguration.clusters']}</value>
                    </set>
                </constructor-arg>

                <!--
                    用于 redis.clients.jedis.JedisCluster.JedisCluster(Set<HostAndPort>, int, int, GenericObjectPoolConfig) 第三个参数 maxRedirections
                    默认值是5
                    一般当此值设置过大时,容易报:Too many Cluster redirections
                -->
                <property name="maxRedirects" value="#{redis['redis.redisClusterConfiguration.maxRedirects']}" />
            </bean>
        </constructor-arg>

        <constructor-arg index="1">
            <bean class="redis.clients.jedis.JedisPoolConfig">
                <property name="maxIdle" value="#{redis['redis.jedisPoolConfig.MaxIdle']}" />
                <property name="testOnBorrow" value="#{redis['redis.jedisPoolConfig.testOnBorrow']}" />
                <property name="testOnReturn" value="#{redis['redis.jedisPoolConfig.testOnReturn']}" />

                <!-- 新版jedis 不支持这个参数了 -->
                <!-- <property name="maxWait" value="#{redis['redis.jedisPoolConfig.MaxWait']}" /> -->
                <!-- <property name="maxActive" value="#{redis['redis.jedisPoolConfig.MaxActive']}" /> -->
            </bean>
        </constructor-arg>
    </bean>

</beans>

redis-cluster.properties

#############for org.springframework.data.redis.connection.RedisClusterConfiguration###################
#只需要配置 master
#理论上只需要配置一个节点即可,配置多个是为了防止单个节点挂掉,
redis.redisClusterConfiguration.clusters=10.88.21.31:10000,10.88.22.25:10000,10.88.21.31:10001

#用于 redis.clients.jedis.JedisCluster.JedisCluster(Set<HostAndPort>, int, int, GenericObjectPoolConfig) 第三个参数 maxRedirections
#默认值是5
#一般当此值设置过大时,容易报:Too many Cluster redirections
redis.redisClusterConfiguration.maxRedirects=3

###########for redis.clients.jedis.JedisPoolConfig##############################

redis.jedisPoolConfig.MaxIdle=60000
redis.jedisPoolConfig.testOnBorrow=true
redis.jedisPoolConfig.testOnReturn=true

#新版jedis 不支持这个参数了
#redis.jedisPoolConfig.MaxActive=50
#redis.jedisPoolConfig.MaxWait=5000

#redis.keystart=gowilddev

启动

成功没有报错

2.2.3 为什么标准的 3主 3从 方案,我们 clusterNodes 仅配置了 3个 master?

  1. 首先配置多个是为了防止单个节点挂掉,理论上只需要配置一个节点即可
  2. 至于为什么不配置slave节点,参见 https://github.com/spring-projects/spring-data-examples spring data 示例中也只配置了master

2.2.4 关于 RedisClusterConfiguration 中的 maxRedirects 属性

重试次数,在执行失败后,进行的重试次数,默认是5, 参见 https://www.zybuluo.com/SailorXiao/note/159072#解读

主要用于 org.springframework.data.redis.connection.jedis.JedisConnectionFactory.createCluster(RedisClusterConfiguration, GenericObjectPoolConfig)

构造 redis.clients.jedis.JedisCluster.JedisCluster(Set, int, int, GenericObjectPoolConfig) 使用的

源码:

    /**
     * Creates {@link JedisCluster} for given {@link RedisClusterConfiguration} and {@link GenericObjectPoolConfig}.
     * 
     * @param clusterConfig must not be {@literal null}.
     * @param poolConfig can be {@literal null}.
     * @return
     * @since 1.7
     */
    protected JedisCluster createCluster(RedisClusterConfiguration clusterConfig, GenericObjectPoolConfig poolConfig) {

        Assert.notNull("Cluster configuration must not be null!");

        Set<HostAndPort> hostAndPort = new HashSet<HostAndPort>();
        for (RedisNode node : clusterConfig.getClusterNodes()) {
            hostAndPort.add(new HostAndPort(node.getHost(), node.getPort()));
        }

        int redirects = clusterConfig.getMaxRedirects() != null ? clusterConfig.getMaxRedirects().intValue() : 5;

        if (StringUtils.hasText(getPassword())) {
            throw new IllegalArgumentException("Jedis does not support password protected Redis Cluster configurations!");
        }

        if (poolConfig != null) {
            return new JedisCluster(hostAndPort, timeout, redirects, poolConfig);
        }
        return new JedisCluster(hostAndPort, timeout, redirects, poolConfig);
    }

一般当此值设置过大时,容易报:Too many Cluster redirections

3.问题

3.1 spring-session>1.2.2.RELEASE 不兼容 spring-data-redis > 1.7.4.RELEASE

目前 spring-session 最新版本是 1.2.2.RELEASE,内置依赖 spring-data-redis 版本是 1.7.1.RELEASE
而目前 spring-data-redis 最新版本是 1.7.4.RELEASE

尝试升级 spring-data-redis>1.7.4.RELEASE 启动报异常

Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sessionRedisTemplate' defined in class path resource [org/springframework/session/data/redis/config/annotation/web/http/RedisHttpSessionConfiguration.class]: Invocation of init method failed; nested exception is java.lang.NoSuchMethodError: org.springframework.core.serializer.support.DeserializingConverter.<init>(Ljava/lang/ClassLoader;)V
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1514)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:521)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:458)
    at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:293)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:223)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:290)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:191)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:921)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:864)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:779)
    at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:817)
    at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:745)
    ... 38 more
Caused by: java.lang.NoSuchMethodError: org.springframework.core.serializer.support.DeserializingConverter.<init>(Ljava/lang/ClassLoader;)V
    at org.springframework.data.redis.serializer.JdkSerializationRedisSerializer.<init>(JdkSerializationRedisSerializer.java:53)
    at org.springframework.data.redis.core.RedisTemplate.afterPropertiesSet(RedisTemplate.java:119)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1573)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1511)
    ... 49 more

4.参考

--to be continued

展开阅读全文

没有更多推荐了,返回首页