关闭

分布式缓存技术redis学习系列(五)——spring-data-redis与JedisPool的区别、使用ShardedJedisPool与spring集成的实现及一致性哈希分析

标签: springredis
4676人阅读 评论(3) 收藏 举报
分类:

一、Redis与spring的整合一般分为spring-data-redis整合和JedisPool整合,先看看两者的区别

1)、引用的依赖不同:

spring-data-redis使用的依赖如下:

    <dependency>  
        <groupId>org.springframework.data</groupId>  
        <artifactId>spring-data-redis</artifactId>  
        <version>1.0.2.RELEASE</version>  
    </dependency>

JedisPool使用的依赖如下:

    <dependency>
        <groupId>redis.clients</groupId>
        <artifactId>jedis</artifactId>
        <version>2.7.2</version>
        <type>jar</type>
        <scope>compile</scope>
    </dependency>

2)、管理Jedis实例方式、操作redis服务的不同:

spring-data-redis:

通过org.springframework.data.redis.connection.jedis.JedisConnectionFactory来管理,即通过工厂类管理,然后通过配置的模版bean,操作redis服务,代码段中充斥大量与业务无关的模版片段代码,代码冗余,不易维护,比如像下面的代码:

    @Autowired
    protected RedisTemplate<Serializable, Serializable> redisTemplate;

    public void saveUser(final User user) {
        redisTemplate.execute(new RedisCallback<Object>() {

            @Override
            public Object doInRedis(RedisConnection connection) throws DataAccessException {
                connection.set(redisTemplate.getStringSerializer().serialize("user.uid." + user.getId()),
                               redisTemplate.getStringSerializer().serialize(user.getName()));
                return null;
            }
        });
    }
@Override
    public User getUser(final long id) {
        return redisTemplate.execute(new RedisCallback<User>() {
            @Override
            public User doInRedis(RedisConnection connection) throws DataAccessException {
                byte[] key = redisTemplate.getStringSerializer().serialize("user.uid." + id);
                if (connection.exists(key)) {
                    byte[] value = connection.get(key);
                    String name = redisTemplate.getStringSerializer().deserialize(value);
                    User user = new User();
                    user.setName(name);
                    user.setId(id);
                    return user;
                }
                return null;
            }
        });
    }

JedisPool方式:

通过redis.clients.jedis.JedisPool来管理,即通过池来管理,通过池对象获取jedis实例,然后通过jedis实例直接操作redis服务,剔除了与业务无关的冗余代码,如下面的代码片段:

    @Autowired
    private JedisPool jedisPool;

    @Override
    public String save(String key,String val) {
        Jedis jedis = jedisPool.getResource();
        return jedis.set(key, val);

    }

从工厂类到池的方式变化,就相当于mybatis连接mysql方变化是一样的,代码变得更简洁,维护也更容易了。

spring-data-redis的集成方式可以查看 java之redis篇(spring-data-redis整合) 这边博文。

但是本文与spring集成并未直接采用JedisPool,而是采用了 ShardedJedisPool ,为什么呢?

因为ShardedJedisPool可以通过一致性哈希实现分布式存储

shared一致性哈希采用以下方案:

1、Redis服务器节点划分:将每台服务器节点采用hash算法划分为160个虚拟节点(可以配置划分权重)

2、将划分虚拟节点采用TreeMap存储

3、对每个Redis服务器的物理连接采用LinkedHashMap存储

4、对Key or KeyTag 采用同样的hash算法,然后从TreeMap获取大于等于键hash值得节点,取最邻近节点存储;当key的hash值大于虚拟节点hash值得最大值时,存入第一个虚拟节点sharded采用的hash算法:MD5 和 MurmurHash两种;默认采用64位的MurmurHash算法;

更为深入解释可以查看 Jedis分片连接池(分布式)jedis源码中ShardedJedis实现sharding文章。

使用非分片的JedisPool和使用分片的ShardedJedisPool示例可以查看 Java内存数据库实践之深入浅出Redis - Jedis分布式(Sharding/Sharded) 这篇文章。

二、使用ShardedJedisPool与spring集成

集成参照:https://my.oschina.net/u/866380/blog/521658

相关依赖jar包

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <spring.version>4.2.6.RELEASE</spring.version>
    </properties>

    <dependencies>

        <!-- spring begin -->
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-context</artifactId>
            <version>${spring.version}</version>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-context-support</artifactId>
            <version>${spring.version}</version>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-test</artifactId>
            <version>${spring.version}</version>
        </dependency>


        <dependency>
            <groupId>redis.clients</groupId>
            <artifactId>jedis</artifactId>
            <version>2.7.2</version>
            <type>jar</type>
        </dependency>

        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.11</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

Spring 配置文件spring-redis.xml

<?xml version="1.0" encoding="UTF-8"?>
<beans  xmlns="http://www.springframework.org/schema/beans" 
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"       
        xmlns:context="http://www.springframework.org/schema/context" 
        xsi:schemaLocation="
            http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-4.0.xsd
            http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-4.0.xsd">

    <!-- 开启注解驱动 -->
    <context:annotation-config></context:annotation-config>

    <!-- 引入属性文件 -->
    <context:property-placeholder location="classpath:spring-redis.properties" ignore-unresolvable="true" />

    <!-- 配置redis数据库连接池 -->
    <bean id="jedisPoolConfig" class="redis.clients.jedis.JedisPoolConfig">
        <property name="maxTotal" value="${redis.pool.maxActive}" />
        <property name="maxIdle" value="${redis.pool.maxIdle}" />
        <property name="minIdle" value="${redis.pool.minIdle}" />
        <property name="maxWaitMillis" value="${redis.pool.maxWait}" />
        <property name="lifo" value="${redis.pool.lifo}" />
        <property name="testOnBorrow" value="${redis.pool.testOnBorrow}" />
    </bean>

    <!-- 配置redis共享连接池,业务层主要通过该bean访问redis服务器 -->
    <bean id="shardedJedisPool" class="redis.clients.jedis.ShardedJedisPool">
        <constructor-arg index="0" ref="jedisPoolConfig" />
        <constructor-arg index="1">
            <list>
                <bean class="redis.clients.jedis.JedisShardInfo">
                    <constructor-arg index="0" value="${redis.uri.0}" />
                </bean>
            </list>
        </constructor-arg>
    </bean>

</beans>

几个注意的点:

(1)如果你有多个数据源需要通过<context:property-placeholder>管理,且不愿意放在一个配置文件里,那么一定要加上ignore-unresolvable=“true”

(2)注意新版的(具体从哪个版本开始不清楚,有兴趣可以查一下)JedisPoolConfig的property name,不是maxActive而是maxTotal,而且没有maxWait属性,建议看一下Jedis源码。

(3)ShardedJedisPool有多种构造函数,选择你需要的(具体看源码),示例中只初始化了一个分片,并使用了通过指定host的构造器(具体格式见下文),如果有集群,在下增加新的即可。

(4)当然,你的spring核心配置文件中得有扫描组件。

(5)客户端分片使用ShardedJedisPool,设置DB和超时时间

redis连接池配置文件spring-redis.properties

#最大分配的对象数
redis.pool.maxActive=10
#最大能够保持idel状态的对象数  
redis.pool.maxIdle=10
#最小空闲的对象数。2.5.1以上版本有效
redis.pool.minIdle=8
#当池内没有返回对象时,最大等待时间
redis.pool.maxWait=1000
#是否启用Lifo。如果不设置,默认为true。2.5.1以上版本有效
redis.pool.lifo=false
#当调用borrow Object方法时,是否进行有效性检查  
redis.pool.testOnBorrow=false

## REDIS URI
## 使用DB0库
##redis://用户名:密码@host:port/库,其中用户名随意填写的
redis.uri.0=redis://testRedis:foobared@119.254.166.136:6379/0

注意:redis uir的格式为 redis://用户名:密码@host:port/库,其中用户名随意填写的,如果没有用户名和密码,格式为redis://:@host:port/库,使用这种方式,配置内容少,还能自定义db index,非常适合开发、测试和线上环境的切换

代码实现

(1)推荐大家使用统一的类来管理Jedis实例的生成和回收,参考代码如下:JedisDataSourceImpl

@Repository("jedisDS")
public class JedisDataSourceImpl implements JedisDataSource {
    private static final Logger LOG = LoggerFactory.getLogger(JedisDataSourceImpl.class);

    @Autowired
    private ShardedJedisPool shardedJedisPool;

    @Override
    public ShardedJedis getRedisClient() {
        ShardedJedis shardJedis = null;
        try {
            shardJedis = shardedJedisPool.getResource();
            return shardJedis;
        } catch (Exception e) {
            LOG.error("[JedisDS] getRedisClent error:" + e.getMessage());
            if (null != shardJedis)
                shardJedis.close();
        }
        return null;
    }

    @Override
    public void returnResource(ShardedJedis shardedJedis) {
        shardedJedis.close();
    }

    @Override
    public void returnResource(ShardedJedis shardedJedis, boolean broken) {
        shardedJedis.close();
    }

}

这里要注意的是Jedis实例的回收,从jedis2.6开始,原returnResource方式已经提示在后续版本中不再支持,所以不建议大家再用ShardedJedisPool里的returnResource和retureBrokenResource方法,虽然在2.7中还支持(毕竟是因为这两个方法存在漏洞)。

(2)编写具体的Jedis操作类(片断):RedisClientTemplate

@Repository("redisClientTemplate")
public class RedisClientTemplate {
    private static final Logger log = LoggerFactory.getLogger(RedisClientTemplate.class);

    @Autowired
    private JedisDataSource redisDataSource;

    public void disconnect() {
        ShardedJedis shardedJedis = redisDataSource.getRedisClient();
        shardedJedis.disconnect();
    }

    /**
     * 设置单个值
     * 
     * @param key
     * @param value
     * @return
     */
    public String set(String key, String value) {
        String result = null;

        ShardedJedis shardedJedis = redisDataSource.getRedisClient();
        if (shardedJedis == null) {
            return result;
        }
        boolean broken = false;
        try {
            result = shardedJedis.set(key, value);
        } catch (Exception e) {
            log.error(e.getMessage(), e);
            broken = true;
        } finally {
            redisDataSource.returnResource(shardedJedis, broken);
        }
        return result;
    }

    /**
     * 获取单个值
     * 
     * @param key
     * @return
     */
    public String get(String key) {
        String result = null;
        ShardedJedis shardedJedis = redisDataSource.getRedisClient();
        if (shardedJedis == null) {
            return result;
        }

        boolean broken = false;
        try {
            result = shardedJedis.get(key);

        } catch (Exception e) {
            log.error(e.getMessage(), e);
            broken = true;
        } finally {
            redisDataSource.returnResource(shardedJedis, broken);
        }
        return result;
    }

    public Boolean exists(String key) {
        Boolean result = false;
        ShardedJedis shardedJedis = redisDataSource.getRedisClient();
        if (shardedJedis == null) {
            return result;
        }
        boolean broken = false;
        try {
            result = shardedJedis.exists(key);
        } catch (Exception e) {
            log.error(e.getMessage(), e);
            broken = true;
        } finally {
            redisDataSource.returnResource(shardedJedis, broken);
        }
        return result;
    }

    public String type(String key) {
        String result = null;
        ShardedJedis shardedJedis = redisDataSource.getRedisClient();
        if (shardedJedis == null) {
            return result;
        }
        boolean broken = false;
        try {
            result = shardedJedis.type(key);

        } catch (Exception e) {
            log.error(e.getMessage(), e);
            broken = true;
        } finally {
            redisDataSource.returnResource(shardedJedis, broken);
        }
        return result;
    }

    /**
     * 在某段时间后失效
     * 
     * @param key
     * @param seconds
     * @return
     */
    public Long expire(String key, int seconds) {
        Long result = null;
        ShardedJedis shardedJedis = redisDataSource.getRedisClient();
        if (shardedJedis == null) {
            return result;
        }
        boolean broken = false;
        try {
            result = shardedJedis.expire(key, seconds);

        } catch (Exception e) {
            log.error(e.getMessage(), e);
            broken = true;
        } finally {
            redisDataSource.returnResource(shardedJedis, broken);
        }
        return result;
    }

    /**
     * 在某个时间点失效
     * 
     * @param key
     * @param unixTime
     * @return
     */
    public Long expireAt(String key, long unixTime) {
        Long result = null;
        ShardedJedis shardedJedis = redisDataSource.getRedisClient();
        if (shardedJedis == null) {
            return result;
        }
        boolean broken = false;
        try {
            result = shardedJedis.expireAt(key, unixTime);

        } catch (Exception e) {
            log.error(e.getMessage(), e);
            broken = true;
        } finally {
            redisDataSource.returnResource(shardedJedis, broken);
        }
        return result;
    }

    public Long ttl(String key) {
        Long result = null;
        ShardedJedis shardedJedis = redisDataSource.getRedisClient();
        if (shardedJedis == null) {
            return result;
        }
        boolean broken = false;
        try {
            result = shardedJedis.ttl(key);

        } catch (Exception e) {
            log.error(e.getMessage(), e);
            broken = true;
        } finally {
            redisDataSource.returnResource(shardedJedis, broken);
        }
        return result;
    }

    public boolean setbit(String key, long offset, boolean value) {

        ShardedJedis shardedJedis = redisDataSource.getRedisClient();
        boolean result = false;
        if (shardedJedis == null) {
            return result;
        }
        boolean broken = false;
        try {
            result = shardedJedis.setbit(key, offset, value);
        } catch (Exception e) {
            log.error(e.getMessage(), e);
            broken = true;
        } finally {
            redisDataSource.returnResource(shardedJedis, broken);
        }
        return result;
    }

    public boolean getbit(String key, long offset) {
        ShardedJedis shardedJedis = redisDataSource.getRedisClient();
        boolean result = false;
        if (shardedJedis == null) {
            return result;
        }
        boolean broken = false;

        try {
            result = shardedJedis.getbit(key, offset);
        } catch (Exception e) {
            log.error(e.getMessage(), e);
            broken = true;
        } finally {
            redisDataSource.returnResource(shardedJedis, broken);
        }
        return result;
    }

    public long setRange(String key, long offset, String value) {
        ShardedJedis shardedJedis = redisDataSource.getRedisClient();
        long result = 0;
        if (shardedJedis == null) {
            return result;
        }
        boolean broken = false;
        try {
            result = shardedJedis.setrange(key, offset, value);
        } catch (Exception e) {
            log.error(e.getMessage(), e);
            broken = true;
        } finally {
            redisDataSource.returnResource(shardedJedis, broken);
        }
        return result;
    }

    public String getRange(String key, long startOffset, long endOffset) {
        ShardedJedis shardedJedis = redisDataSource.getRedisClient();
        String result = null;
        if (shardedJedis == null) {
            return result;
        }
        boolean broken = false;
        try {
            result = shardedJedis.getrange(key, startOffset, endOffset);

        } catch (Exception e) {
            log.error(e.getMessage(), e);
            broken = true;
        } finally {
            redisDataSource.returnResource(shardedJedis, broken);
        }
        return result;
    }
}

(3)好了,接下来在你的业务代码里加载RedisClientTemplate.class就可以了。


@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration("classpath:spring-redis.xml")
public class RedisTest{
    @Autowired
    private RedisClientTemplate redisClientTemplate ;

    @Test
    public void testSave(){
        String key = "10086";
        String val = "中国移动";
        String r = redisClientTemplate .save(key,val);
        System.out.println(r);
    }

    @Test
    public void testFind(){
        String key = "10086";
        String result = redisClientTemplate .find(key);
        System.out.println(result);
    }

    @Test public void testDel(){
        String key = "10086";
        long l = redisClientTemplate .delete(key);
        System.out.println(l);
    }
}

补充:


1、shardedjedispool分片实现集群与jediscluster实现集群区别以及jediscluster实现集群可以查看 分布式缓存技术redis学习系列(七)——spring整合jediscluster

2、shardedjedispool分片集群与jediscluster集群同样不具有高可用特性,但基于shardedjedispool的ShardedJedisSentinelPool可以实现集群高可用,redis服务一台master挂机,sentinel哨兵进行主从切换之后,客户端程序自动完成同步的主从切换。
具体的实现方式就是通过sentinel的sentinelGetMasterAddrByName(masterName);(实际执行sentinel的get-master-addr-by-name命令)获取哨兵监控的所有master节点;然后用一个MasterListener启用一个线程,利用redis的订阅与发布机制,订阅+switch-master频道,时时获取redis服务端主从切换的信息。
实现原理和源码分析可以查看 基于Redis Sentinel的Redis集群(主从&Sharding)高可用方案 相关源码地址位于 sharded-jedis-sentinel-pool

1
0

查看评论
* 以上用户言论只代表其个人观点,不代表CSDN网站的观点或立场
    个人资料
    • 访问:707598次
    • 积分:7951
    • 等级:
    • 排名:第2691名
    • 原创:201篇
    • 转载:69篇
    • 译文:0篇
    • 评论:83条
    文章分类
    最新评论