[工具类] 系列二 Lettuce 访问Redis 工具类 RedisUtil

最近公司Redis集群启用了ssl和密码校验，使用Jedis访问Redis Cluster的时候，支持不太好。看到spring-data-redis 2.x开始使用Lettuce访问Redis，于是开始吃个螃蟹。

 

Lettuce介绍

https://lettuce.io/docs/getting-started.html

https://lettuce.io/core/release/reference/

https://www.cnblogs.com/throwable/p/11601538.html

 

版本介绍

5.0 以后移到io.lettuce包里了

io.lettuce » lettuce-core

4.0以前，在biz.paluch.redis包里。

biz.paluch.redis » lettuce

 

我使用的版本

// for redis access
compile ('biz.paluch.redis:lettuce:4.2.2.Final')
compile 'io.reactivex:rxjava:1.1.9'

 

这里就是spark 环境里单线程使用，没有使用线程安全的连接池。代码如下：

/**
 * Use Lettuce as client to access Redis. <br/>
 * https://lettuce.io/docs/getting-started.html <br />
 *
 * @author adore.chen
 * @date 2020-04-29
 */
public class RedisUtil {

    private static class Helper {

        static RedisClusterClient redisClient;

        static StatefulRedisClusterConnection<String, String> connection;

        static final MessageFormat KEY_FORMAT = new MessageFormat(KEY_SCHEMA);

        static {
            // get password from Vault
            String password = VaultRead.get(REDIS_AUTH);

            RedisURI uri = RedisURI.builder()
                    .withHost(Config.getString(REDIS_HOST))
                    .withPort(Config.getInt(REDIS_PORT))
                    .withPassword(password)
                    .withSsl(true)
                    .build();

           

            redisClient = RedisClusterClient.create(uri);
            connection = redisClient.connect();

            System.out.println("==> Connected to Redis");

            Runtime.getRuntime().addShutdownHook(new Thread() {
                public void run() {
                    if (connection != null) {
                        connection.close();
                    }
                    if (redisClient != null) {
                        redisClient.shutdown();
                    }
                }
            });
        }

    }

    public static String getKey(String memberSrl, String value) {
        return Helper.KEY_FORMAT.format(new String[]{memberSrl, value});
    }

    /**
     * save the expiration value if more than stored value.
     * @param key     key
     * @param value     value
     * @param expireTime    expire time in milliseconds.
     */
    public static String set(String key, String value, long expireTime) {

        RedisAdvancedClusterCommands<String, String> commands = Helper.connection.sync();

        String key = getKey(memberSrl, value);
        String oldExpire = commands.get(key);

        return commands.psetex(key, ttl, expireTime + "");

    }

    /**
     * set all key values, You should ensure that the expiration time
     * should be greater than the value that redis has stored. <br />
     * key from KEY_FORMAT.format(memberSrl, uuid/pcid/ip/fp) <br />
     *
     * @param keyVals
     * @return
     */
    public static String sets(Map<String, String> keyVals) {
        if (keyVals == null || keyVals.isEmpty()) {
            return "Empty Parameters";
        }

        return Helper.connection.sync().mset(keyVals);
    }

    /**
     * get the expiration time of input key.
     * @param memberSrl
     * @param value
     * @return
     */
    public static Long get(String memberSrl, String value) {
        String key = getKey(memberSrl, value);
        String expire = Helper.connection.sync().get(key);
        return (expire == null) ? 0 : Long.parseLong(expire);
    }

    /**
     * input many keys and then query all
     * @param keys
     * @return
     */
    public static List<String> gets(String... keys) {
        return Helper.connection.sync().mget(keys);
    }

    /**
     *
     * @param memberSrl
     * @param categoryValue
     * @return
     */
    public static Long del(String memberSrl, String categoryValue) {
        String key = getKey(memberSrl, categoryValue);
        return Helper.connection.sync().del(key);
    }

    /**
     * delete many keys, key from KEY_FORMAT.format(memberSrl, uuid/pcid/ip/fp);
     * @param keys
     * @return
     */
    public static Long dels(String... keys) {
        if (keys == null || keys.length == 0) {
            return 0L;
        }

        return Helper.connection.sync().del(keys);
    }

    /**
     * get lettuce connection, you can do command on it.
     * @return
     */
    public static StatefulRedisClusterConnection getConnection() {
        return Helper.connection;
    }

    private RedisUtil() {

    }

}

 

若是有多线程需要，可使用apache commons.pool2 来封装。参考代码如下：

/**
 * lettuce-core 5.1.1
 * commons-pool2  2.6.0 （2.6.0版本以上的版本，否则编译不通过）
 */
public class RedisStreamDemo {
 
    public static void main(String[] args) {
 
        List<RedisURI> list = new ArrayList<>();
        list.add(RedisURI.create("redis://192.168.2.4:7000"));
        list.add(RedisURI.create("redis://192.168.2.5:7000"));
        list.add(RedisURI.create("redis://192.168.2.6:7000"));
        list.add(RedisURI.create("redis://192.168.2.4:7001"));
        list.add(RedisURI.create("redis://192.168.2.5:7001"));
        list.add(RedisURI.create("redis://192.168.2.6:7001"));
        RedisClusterClient clusterClient = RedisClusterClient.create(list);
        //集群Redis
        RedisClusterClient client = RedisClusterClient.create(list);
        GenericObjectPool<StatefulRedisClusterConnection<String, String>> pool;
        GenericObjectPoolConfig<StatefulRedisClusterConnection<String, String>> poolConfig = new GenericObjectPoolConfig();
        poolConfig.setMinIdle(8);
        poolConfig.setMaxIdle(8);
        poolConfig.setMaxTotal(16);
        poolConfig.setMinEvictableIdleTimeMillis(1000*30);
        poolConfig.setSoftMinEvictableIdleTimeMillis(1000*30);
        poolConfig.setMaxWaitMillis(0);
        pool = ConnectionPoolSupport.createGenericObjectPool(() -> {
           System.err.println("Requesting new StatefulRedisClusterConnection "+System.currentTimeMillis());
            return client.connect();
        }, poolConfig);
 
        StatefulRedisClusterConnection<String, String> connection = null;
        try {
            connection = pool.borrowObject();
            connection.setReadFrom(ReadFrom.MASTER_PREFERRED);
 
            RedisAdvancedClusterAsyncCommands<String, String> commands = connection.async();
            commands.set("id","taozhongyu");
            RedisFuture<String> future = commands.get("id");
            String str = future.get();
            System.out.println(str);
 
        } catch (Exception e) {
            e.printStackTrace();
        }
        pool.close();
        clusterClient.shutdown();    
 
    }
}

 

 

开发过程中遇到的问题：

异常信息：

io.lettuce.core.RedisException: Cannot retrieve initial cluster partitions from initial URIs [RedisURI [host=‘192.168.1.1’, port=6379]]

 java.lang.LinkageError: loader constraint violation: when resolving method "io.netty.channel.group.DefaultChannelGroup.<init>(Lio/netty/util/concurrent/EventExecutor;)V" the class loader (instance of org/apache/spark/util/ChildFirstURLClassLoader) of the current class, io/lettuce/core/AbstractRedisClient, and the class loader (instance of sun/misc/Launcher$AppClassLoader) for the method's defining class, io/netty/channel/group/DefaultChannelGroup, have different Class objects for the type io/netty/util/concurrent/EventExecutor used in the signature

这两个异常基本上都是包冲突导致的。

1. netty 包冲突

由于环境中用到HBase Client的版本为

// for hbase access
compile "org.apache.hbase:hbase-client:1.2.0-cdh5.10.1"
compile "org.apache.hbase:hbase-server:1.2.0-cdh5.10.1"

这里面的netty包为4.0.23.Final ，使用Lettcue 5 + 系列（ netty版本为4.1+ ）会导致版本冲突，必须降级netty版本和Hbase保持一致。

io.netty:netty-all:4.0.23.Final
+--- org.apache.hadoop:hadoop-hdfs:2.6.0-cdh5.10.1
|    \--- org.apache.hbase:hbase-server:1.2.0-cdh5.10.1
|         \--- compile
+--- org.apache.hbase:hbase-client:1.2.0-cdh5.10.1
|    +--- compile
|    \--- org.apache.hbase:hbase-server:1.2.0-cdh5.10.1 (*)
+--- org.apache.hbase:hbase-prefix-tree:1.2.0-cdh5.10.1
|    \--- org.apache.hbase:hbase-server:1.2.0-cdh5.10.1 (*)
\--- org.apache.hbase:hbase-server:1.2.0-cdh5.10.1 (*)
 

2.  reactor-core 包冲突

gradle 环境使用命令检查包依赖：

../gradlew dependencyInsight --configuration compile --dependency reactor-core

 

参考文章：

https://lettuce.io/docs/getting-started.html

https://lettuce.io/core/release/reference/

https://www.cnblogs.com/throwable/p/11601538.html

Lettuce Pool Connect：https://blog.csdn.net/xiyujianxia/article/details/83275403