Redis 6.2的过期删除策略源码分析(惰性删除)

都知道redis采用的过期删除策略是定期删除和惰性删除，对于这l两个的解释可以看一下Redis 键的过期删除策略及缓存淘汰策略
下面是根据翻译软件和自己的理解翻译过来的，英文原文也在上面，如果不清楚或者不对可以看一下英文
定时删除的源码可以看一下我的另一篇文章Redis的过期删除策略源码分析(定时删除)

1、当查询key的源码执行逻辑

你会疑问为什么标题明明是惰性删除策略，而这里是查询key的执行的逻辑呢？
如果你看过上面的链接你就不会这么问了，那是因为惰性删除策略只有你在查询或者操作时才执行key是否过期，所以看下去就对了

/* Lookup a key for read operations, or return NULL if the key is not found
 * in the specified DB.
 * 为读取操作查找键，如果在指定的数据库中找不到该键，则返回NULL
 * As a side effect of calling this function:
 * 1. A key gets expired if it reached it's TTL.
 * 2. The key last access time is updated.
 * 3. The global keys hits/misses stats are updated (reported in INFO).
 * 4. If keyspace notifications are enabled, a "keymiss" notification is fired.
 * 作为调用此函数的副作用：
 * 1.如果一个密钥到达了它的TTL，它就会过期。
 * 2.更新密钥上次访问时间。
 * 3.更新全局键命中/未命中统计（在INFO中报告）。
 * 4.如果启用了键空间通知，则会触发“keymiss”通知。
 * This API should not be used when we write to the key after obtaining
 * the object linked to the key, but only for read only operations.
 *  当我们在获取链接到键的对象后写入键时，不应使用此API，而应仅用于只读操作
 * Flags change the behavior of this command:
 *  标志更改此命令的行为
 *  LOOKUP_NONE (or zero): no special flags are passed.
 * LOOKUP_NONE (or zero)：不传递特殊标志。
 *  LOOKUP_NOTOUCH: don't alter the last access time of the key.
 *  LOOKUP_NOTOUCH: 不要更改key的最后访问时间
 * Note: this function also returns NULL if the key is logically expired
 * but still existing, in case this is a slave, since this API is called only
 * for read operations. Even if the key expiry is master-driven, we can
 * correctly report a key is expired on slaves even if the master is lagging
 * expiring our key via DELs in the replication link.
 *注意：如果密钥在逻辑上过期，此函数还返回NULL
 *但仍然存在，以防这是一个从属，因为这个API只被调用
 *用于读取操作。即使key过期是主驱动的，我们也可以
 *正确报告从机上的key过期，即使主机延迟
 *通过复制链接中的DELs使密钥过期
 */
robj *lookupKeyReadWithFlags(redisDb *db, robj *key, int flags) {
    robj *val;

    if (expireIfNeeded(db,key) == 1) { //这个方法是关键点
        /* If we are in the context of a master, expireIfNeeded() returns 1
         * when the key is no longer valid, so we can return NULL ASAP. */
        if (server.masterhost == NULL)
            goto keymiss;

        /* However if we are in the context of a slave, expireIfNeeded() will
         * not really try to expire the key, it only returns information
         * about the "logical" status of the key: key expiring is up to the
         * master in order to have a consistent view of master's data set.
         *
         * However, if the command caller is not the master, and as additional
         * safety measure, the command invoked is a read-only command, we can
         * safely return NULL here, and provide a more consistent behavior
         * to clients accessing expired values in a read-only fashion, that
         * will say the key as non existing.
         *
         * Notably this covers GETs when slaves are used to scale reads. */
        if (server.current_client &&
            server.current_client != server.master &&
            server.current_client->cmd &&
            server.current_client->cmd->flags & CMD_READONLY)
        {
            goto keymiss;
        }
    }
    val = lookupKey(db,key,flags);
    if (val == NULL)
        goto keymiss;
    server.stat_keyspace_hits++;
    return val;

keymiss:
    if (!(flags & LOOKUP_NONOTIFY)) {
        notifyKeyspaceEvent(NOTIFY_KEY_MISS, "keymiss", key, db->id);
    }
    server.stat_keyspace_misses++;
    return NULL;
}

这里有一个关键的方法就是expireIfNeeded，这个是判断这个key是否过期的方法

2、expireIfNeeded(判断key是否过期)

/* This function is called when we are going to perform some operation
 * in a given key, but such key may be already logically expired even if
 * it still exists in the database. The main way this function is called
 * is via lookupKey*() family of functions.
 *当我们要在给定的key中执行某些操作时，会调用此函数，因为这样的key可能已经在逻辑上过期，即使它仍然存在于数据库中。
 *调用此函数的主要方式是通过lookupKey*（）函数族
 *
 * The behavior of the function depends on the replication role of the
 * instance, because slave instances do not expire keys, they wait
 * for DELs from the master for consistency matters. However even
 * slaves will try to have a coherent return value for the function,
 * so that read commands executed in the slave side will be able to
 * behave like if the key is expired even if still present (because the
 * master has yet to propagate the DEL).
 *函数的行为取决于实例的复制角色，因为从属实例不会使密钥过期，所以它们会等待来自主实例的del以确保一致性。
 *然而，即使从机也会尝试为函数提供一个一致的返回值，这样在从机端执行的read命令将能够像密钥过期一样工作，即使key仍然存在（因为主机尚未传播DEL）
 *
 * In masters as a side effect of finding a key which is expired, such
 * key will be evicted from the database. Also this may trigger the
 * propagation of a DEL/UNLINK command in AOF / replication stream.
 *在masters中，作为查找过期key的副作用，此类key将从数据库中逐出。
 *这也可能触发AOF/复制流中DEL/UNLINK命令的传播
 *
 * The return value of the function is 0 if the key is still valid,
 * otherwise the function returns 1 if the key is expired.
 *如果key仍然有效，则函数返回值为 0，否则如果key已过期，则函数返回 1
 */
int expireIfNeeded(redisDb *db, robj *key) {
    if (!keyIsExpired(db,key)) return 0;

    /* If we are running in the context of a slave, instead of
     * evicting the expired key from the database, we return ASAP:
     * the slave key expiration is controlled by the master that will
     * send us synthesized DEL operations for expired keys.
     * 如果我们在一个 slave 的上下文中运行,而不是从database中驱逐过期的key,我们尽快返回
       slave key 的过期时间由 master 控制,将向我们发送过期key的 DEL 操作

     * Still we try to return the right information to the caller,
     * that is, 0 if we think the key should be still valid, 1 if
     * we think the key is expired at this time.
        我们仍然尝试将正确的信息返回给调用者,也就是说如果是0，我们认为key应该仍然有效
        如果是 1，我们认为此时密钥已过期
     */

    if (server.masterhost != NULL) return 1;

    /* If clients are paused, we keep the current dataset constant,
     * but return to the client what we believe is the right state. Typically,
     * at the end of the pause we will properly expire the key OR we will
     * have failed over and the new primary will send us the expire. */
     /*如果客户端暂停，我们将保持当前数据集不变，但将我们认为正确的状态返回给客户端
     *通常,在暂停结束时，我们将正确地使key过期,或者我们将失败,新的主节点向我们发送过期信息
     */
    if (checkClientPauseTimeoutAndReturnIfPaused()) return 1;
    /*删除key,lazyfree_lazy_expire 是Redis的配置项之一，它的作用是是否开启惰性删除 (默认不开启)，*/
    /* Delete the key */
    if (server.lazyfree_lazy_expire) {
         //如果是惰性删除，走的是异步删除
        dbAsyncDelete(db,key);
    } else {
      //否则走的是同步删除
        dbSyncDelete(db,key);
    }
    //统计过期 keys的数量
    server.stat_expiredkeys++;
    //向其他salve节点传播过期的key
    propagateExpire(db,key,server.lazyfree_lazy_expire);
    notifyKeyspaceEvent(NOTIFY_EXPIRED,
        "expired",key,db->id);
    //每次修改数据库中的键时，都会调用函数 signalModifiedKey(),如果某个key被监视，下次执行取消监视
    signalModifiedKey(NULL,db,key);
    return 1;
}

这里判断完这个是否过期就去删除这个key了，所以这个方法即是判断key是否过期的方法，里面也加了删除key的方法，虽然这个删除不是彻底删除

在代码上也能看出对于从节点判断过期是有快速失败的，只有主节点才能判断key是否过期，这样master->salve的执行链就可以保证

server.lazyfree_lazy_expire 是redis的配置，来指定查询到key过期时是采用同步的方式还是异步的方式来删除key，下面有同步删除的源码和异步删除的源码

看到这里其实你也就理解了，不管设不设置定时删除策略，在查询时都会有惰性删除策略，这个是写在代码里的，不是配置的，关键就在expireIfNeeded方法

3、惰性删除方法源码

/* Delete a key, value, and associated expiration entry if any, from the DB.
 * If there are enough allocations to free the value object may be put into
 * a lazy free list instead of being freed synchronously. The lazy free list
 * will be reclaimed in a different bio.c thread.
 *如果存在则从数据库中删除key、值和相关过期条目，
 *如果有足够的分配来释放值对象，则可以将其放入延迟释放列表中，而不是同步释放
 *空闲列表将在另一个bio.c线程中回收。
 */
#define LAZYFREE_THRESHOLD 64
int dbAsyncDelete(redisDb *db, robj *key) {
    /* Deleting an entry from the expires dict will not free the sds of
     * the key, because it is shared with the main dictionary.
     * 从过期 dict中删除条目不会释放key的sds，因为它与主词典共享
     */
    if (dictSize(db->expires) > 0) dictDelete(db->expires,key->ptr);

    /* If the value is composed of a few allocations, to free in a lazy way
     * is actually just slower... So under a certain limit we just free
     * the object synchronously.
      *如果值由几个或多个组成，以懒惰的方式释放实际上只是速度较慢。。。因此，在一定的限制下，我们只是同步释放对象
      **/
    dictEntry *de = dictUnlink(db->dict,key->ptr);
    if (de) {
        robj *val = dictGetVal(de);

        /* Tells the module that the key has been unlinked from the database.
        * 告诉模块，这个key已从数据库中取消连接
        */
        moduleNotifyKeyUnlink(key,val);

        size_t free_effort = lazyfreeGetFreeEffort(key,val);

        /* If releasing the object is too much work, do it in the background
         * by adding the object to the lazy free list.
         * Note that if the object is shared, to reclaim it now it is not
         * possible. This rarely happens, however sometimes the implementation
         * of parts of the Redis core may call incrRefCount() to protect
         * objects, and then call dbDelete(). In this case we'll fall
         * through and reach the dictFreeUnlinkedEntry() call, that will be
         * equivalent to just calling decrRefCount().
         *如果释放对象的工作量太大，可以在后台通过将对象添加到空闲列表来完成。注意，如果对象是共享的，那么现在要回收它就不是了
         * 可能的这种情况很少发生，但是有时候Redis核心部分的实现可能会调用incrRefCount（）来保护对象，然后调用dbDelete（）。
         * 在本例中，我们将完成并到达dictFreeUnlinkdentry（）调用，这相当于只调用decrefCount（）。
         */
        if (free_effort > LAZYFREE_THRESHOLD && val->refcount == 1) {
            atomicIncr(lazyfree_objects,1);
            bioCreateLazyFreeJob(lazyfreeFreeObject,1, val);
            dictSetVal(db->dict,de,NULL);
        }
    }

    /* Release the key-val pair, or just the key if we set the val
     * field to NULL in order to lazy free it later.
     *释放key-val对，或者如果我们将val字段设置为NULL，则只释放key，以便稍后延迟释放它
     */
    if (de) {
        dictFreeUnlinkedEntry(db->dict,de);
        if (server.cluster_enabled) slotToKeyDel(key->ptr);
        return 1;
    } else {
        return 0;
    }
}

4、同步删除方法源码

/* Delete a key, value, and associated expiration entry if any, from the DB
*如果存在则从数据库中删除key、值和相关过期条目，
*/
int dbSyncDelete(redisDb *db, robj *key) {
    /* Deleting an entry from the expires dict will not free the sds of
     * the key, because it is shared with the main dictionary.
     *从过期 dict中删除条目不会释放key的sds，因为它与主词典共享
      */
    if (dictSize(db->expires) > 0) dictDelete(db->expires,key->ptr);
    dictEntry *de = dictUnlink(db->dict,key->ptr);
    if (de) {
        robj *val = dictGetVal(de);
        /* Tells the module that the key has been unlinked from the database.
        * 告诉模块，这个key已从数据库中取消连接
        */
        moduleNotifyKeyUnlink(key,val);
        dictFreeUnlinkedEntry(db->dict,de);
        if (server.cluster_enabled) slotToKeyDel(key->ptr);
        return 1;
    } else {
        return 0;
    }
}

5、用户命令发出删除key时的redis如何选择删除

既然讲到同步异步删除key的方法了，那如果用户主动去删除key会有什么样的执行逻辑呢？

void delCommand(client *c) {
    //lazyfree_lazy_user_del 是否启动懒惰删除
    delGenericCommand(c,server.lazyfree_lazy_user_del);
}

void unlinkCommand(client *c) {
    delGenericCommand(c,1);
}

这两个方法最终调用的都是delGenericCommand(client *c, int lazy),只是unlinkCommand默认lazy是1，走的是异步删除key的方式，而delCommand需要读取配置文件，删除采用的是同步的还是异步(懒惰)的方式需要由配置文件中的参数server.lazyfree_lazy_user_del 决定

/* This command implements DEL and LAZYDEL.
 * 此命令实现DEL和懒惰的DEL
*/
void delGenericCommand(client *c, int lazy) {
    int numdel = 0, j;

    for (j = 1; j < c->argc; j++) {
        expireIfNeeded(c->db,c->argv[j]);
        int deleted  = lazy ? dbAsyncDelete(c->db,c->argv[j]) :
                              dbSyncDelete(c->db,c->argv[j]);
        if (deleted) {
            signalModifiedKey(c,c->db,c->argv[j]);
            notifyKeyspaceEvent(NOTIFY_GENERIC,
                "del",c->argv[j],c->db->id);
            server.dirty++;
            numdel++;
        }
    }
    addReplyLongLong(c,numdel);
}