删除bigkey的正确姿势(使用jedis和redisTemplate实现)

对于bigkey来说，我们不能直接使用del删除，避免删除时间过长，导致阻塞其他客户端命令，正确的姿势应该是采用hsan、sscan、zscan渐进式删除，同时我们还要防止bigkey过期时间自动删除的问题（如一个200万的zset设置1小时过期，会触发del操作，造成阻塞，且该操作不会出现在慢查询中）

基于Jedis客户端实现渐进式删除

1、 Hash删除: hscan + hdel

public static void delBigHash(String host , int port , String bigHashKey){
    Jedis jedis = new Jedis(host, port);
    ScanParams scanParams = new ScanParams().count(100);
    String cursor = "0";
    do {
        ScanResult<Map.Entry<String, String>> scanResult = jedis.hscan(bigHashKey, cursor, scanParams);
        List<Map.Entry<String, String>> entryList = scanResult.getResult();
        for (Map.Entry<String, String> entry : entryList) {
            jedis.hdel(bigHashKey, entry.getKey());
        }
        cursor = scanResult.getCursor();
    }while (!"0".equals(cursor));
}

2、List删除: ltrim

public static void delBigList(String host , int port , String bigListKey){
    Jedis jedis = new Jedis(host, port);
    Long llen = jedis.llen(bigListKey); //队列中的总数量
    int counter = 0;                    //记录已经删除的数量
    int size = 100;                     //每次截掉(删除)的数量
    while (counter < llen){
        //每次从左侧删除100个数
        jedis.ltrim(bigListKey, size, llen);
        counter += size;
    }
    //最终删除key
    jedis.del(bigListKey);
}

3、Set删除: sscan + srem

public static void delBigSet(String host , int port , String bigSetKey){
    Jedis jedis = new Jedis(host, port);
    ScanParams scanParams = new ScanParams().count(100);
    String cursor = "0";
    do {
        ScanResult<String> scanResult = jedis.sscan(bigSetKey, cursor,scanParams);
        List<String> memberList = scanResult.getResult();
        for (String member : memberList) {
            jedis.srem(bigSetKey, member);
        }
        cursor = scanResult.getCursor();
    }while (!"0".equals(cursor));

    //删除bigkey
    jedis.del(bigSetKey);
}

4、SortedSet删除: zscan + zrem

public static void delBigZSet(String host , int port , String bigZSetKey){
    Jedis jedis = new Jedis(host, port);
    ScanParams scanParams = new ScanParams().count(100);
    String cursor = "0";
    do {
        ScanResult<Tuple> zscanResult = jedis.zscan(bigZSetKey, cursor,scanParams);
        List<Tuple> tupleList = zscanResult.getResult();
        for (Tuple tuple : tupleList) {
            jedis.zrem(bigZSetKey, tuple.getElement());
        }
        cursor = zscanResult.getCursor();
    }while (!"0".equals(cursor));

    //删除bigkey
    jedis.del(bigZSetKey);
}

在实际开发中，我们可能整合了Spring Boot，使用StringRedisTemplate来操作redis，接下来就来看看StringRedisTemplate如何实现渐进式删除。

基于StringRedisTemplate实现渐进式删除

1、hash渐进式删除

public void delBigHash() throws IOException {
    String bigHashKey = "bigHashKey";
    ScanOptions scanOptions = ScanOptions.scanOptions().count(100).build();
    Cursor<Map.Entry<Object, Object>> cursor = redisTemplate.opsForHash().scan(bigHashKey, scanOptions);
    while (cursor.hasNext()){
        Map.Entry<Object, Object> next = cursor.next();
        redisTemplate.opsForHash().delete(bigHashKey, next.getKey());
    }
    cursor.close();
}

等价于：

public void delBigHash() throws IOException {
    String bigHashKey = "bigHashKey";
    ScanOptions scanOptions = ScanOptions.scanOptions().count(100).build();
    Cursor<Map.Entry<Object, Object>> cursor = redisTemplate.opsForHash().scan(bigHashKey, scanOptions);
    //java8新特性
    cursor.forEachRemaining(member -> {
        redisTemplate.opsForHash().delete(bigHashKey,  member.getKey());
    });
   	cursor.close();
}

• cursor.close(); 游标一定要关闭，不然连接会一直增长**；可以使用client lists、info clients、info stats 命令查看客户端连接状态，会发现scan操作一直存在。
• 代码虽然只是调用一次scan方法，但是spring-data-redis已经对scan做了封装，这个scan结合cursor.hasNext会多次redis scan，最终拿到所有match的结果。

2、list渐进式删除

public void delList(){
    Long size = redisTemplate.opsForList().size("bigListKey");
    Long counter = 0L;
    int trimNum = 100;  //每次删除的个数
    while (counter < size){
        redisTemplate.opsForList().trim("bigListKey", counter, trimNum);
        counter += trimNum;
    }
}

3、set渐进式删除

public void delSet() throws IOException {
    String bigSetKey = "bigSetKey";
    ScanOptions scanOptions = ScanOptions.scanOptions().count(100).build();
    Cursor<String> cursor = redisTemplate.opsForSet().scan(bigSetKey, scanOptions);
    while (cursor.hasNext()){
          redisTemplate.opsForSet().remove(bigSetKey, cursor.next());
    }
    cursor.close();
}

等价于：

public void delSet() throws IOException {
    String bigSetKey = "bigSetKey";
    ScanOptions scanOptions = ScanOptions.scanOptions().count(100).build();
    Cursor<String> cursor = redisTemplate.opsForSet().scan(bigSetKey, scanOptions);
    //java8新特性
    cursor.forEachRemaining(member -> {
        redisTemplate.opsForSet().remove(bigSetKey, member);
    });
    cursor.close();
}

4、ZSet渐进式删除

public void delZSet() throws IOException {
    String bigZSetKey = "bigZSetKey";
    ScanOptions scanOptions = ScanOptions.scanOptions().count(100).build();
    Cursor<ZSetOperations.TypedTuple<String>> cursor = redisTemplate.opsForZSet().scan(bigZSetKey, scanOptions);
    while (cursor.hasNext()){
        redisTemplate.opsForZSet().remove(bigZSetKey, cursor.next().getValue());
    }
    cursor.close();
}

等价于：

public void delZSet() throws IOException {
    String bigZSetKey = "bigZSetKey";
    ScanOptions scanOptions = ScanOptions.scanOptions().count(100).build();
    Cursor<ZSetOperations.TypedTuple<String>> cursor = redisTemplate.opsForZSet().scan(bigZSetKey, scanOptions);
    cursor.forEachRemaining(member -> {
        redisTemplate.opsForZSet().remove(bigZSetKey, member.getValue());
    });
    cursor.close();
}

参考资料：
https://www.jianshu.com/p/4c842c41ba41