[redis源码]底层文件删除原理

目的

redis中有一些临时文件，需要重命名；如果直接删除文件，（由于是I/O操作）可能会阻塞进程；redis中做了一些“特殊”的处理。

视频

https://study.163.com/instructor/1385962921.htm

man

rename() renames a file, moving it between directories if required. Any other hard links to the file (as created using link(2)) are unaffected. Open file descriptors for oldpath are also unaffected.

close() closes a file descriptor, so that it no longer refers to any file and may be reused. Any record locks (see fcntl(2)) held on the file it was associated with, and owned by the process, are removed (regardless of the file descriptor that was used to obtain the lock).
If fd is the last file descriptor referring to the underlying open file description (see open(2)), the resources associated with the open file description are freed; if the descriptor was the last reference to a file which has been removed using unlink(2) the file is deleted.

unlink() deletes a name from the file system. If that name was the last link to a file and no processes have the file open the file is deleted and the space it was using is made available for reuse.

解释

• rename 目标文件有可能被删除，会调用unlink来删除文件；原文件打开的描述符不受影响。
• close 只是将断开了文件描述符与文件的关系（引用计数减1）。如果文件已经被删除，而且是最后的引用，则会调用unlink来删除文件。
• unlink 可能会将文件删除。

总结

• 引用计数不为0，不删除文件

实现

• 巧妙也利用了操作系统底层文件引用计算原理；也就是如果文件引用计数大于1，执行删除操作，操作系统底层并不会真正删除磁盘上的文件，而且将引用计数减1；直到引用计数为0时，才真正将文件删除。
• 基于这样的一个事实，redis在重命名文件时（重命名文件，目标文件可能会被删除），先调用了一次open，将引用计数加1；后台再通过I/O线程，异步将文件删除；这样就不会阻塞redis的主逻辑线程（因为redis本身是单线程）。

例子

下面有个例子，演示了rename的消耗时间，你可以通过注释open和开启open，来查看消耗的时间；从而验证这个结论。

#include <sys/types.h>
#include <sys/stat.h>
#include <sys/time.h>
#include <fcntl.h>
#include <unistd.h>
#include <errno.h>
#include <stdio.h>
#include <string.h>
#include <stdint.h>

#define _EXIT(format,args...) \
{ \
    printf(format,##args); \
    printf("\n"); \
    _exit(1); \
}

uint64_t milliSecond()
{
	static struct timeval tv;
	gettimeofday(&tv, NULL);
	return (uint64_t)(tv.tv_sec * 1000 + tv.tv_usec * 0.001);
}

void testWriteRead(int fd, const char *file_name, const char *val)
{
    char buf[16];
    int n = read(fd,buf,4);
    if (n > 0)
    {
        printf("test 1:%s\n", buf);
    }
    else _EXIT("can not read '%s' ", file_name);

    n = write(fd,val,strlen(val));
    if (n <= 0) _EXIT("can not write '%s'", file_name);
}


int main()
{
    const char *old_file = "old.redis";
    const char *new_file = "new.redis";
    // test 1 原文件打开的描述符不受影响
    // int fd_old = open(old_file,O_RDWR|O_APPEND);
    // int fd_new = open(new_file,O_WRONLY|O_APPEND|O_CREAT);
    // if (fd_old == -1) _EXIT("can not open 'old.redis'");
    // if (fd_new == -1) _EXIT("can not open 'new.redis'");

    // rename(old_file, new_file);
    // testWriteRead(fd_old, old_file, "abcdefg");
    // testWriteRead(fd_new, new_file, "hijklmn");

    // test 2 删除文件，阻塞进程
    int test_times = 100;

    int buf_size = 1024 * 1024;
    char *buf = new char[buf_size];
    for (int i = 0; i < buf_size; ++i)
    {
        buf[i] = (i + 1) % 128;
    }

    char old_files[32] = {0};
    old_files[0] = '0';
    old_files[1] = '0';

    memcpy(old_files + 2, old_file, strlen(old_file));

    char new_files[32] = {0};
    new_files[0] = '0';
    new_files[1] = '0';
 
    memcpy(new_files + 2, new_file, strlen(new_file));

    for (int i = 0; i < test_times; ++i)
    {
        char prefix1 = i / 10 + '0';
        char prefix2 = i % 10 + '0';

        old_files[0] = prefix1;
        old_files[1] = prefix2;

        new_files[0] = prefix1;
        new_files[1] = prefix2;

        int fd_old = open(old_files,O_WRONLY|O_APPEND|O_CREAT);
        write(fd_old,"123456",6);
        close(fd_old);

        int fd_new = open(new_files,O_WRONLY|O_APPEND|O_CREAT);
        write(fd_new,buf,buf_size);
        close(fd_new);
    }

    delete [] buf;
    
    int fd_old = open(old_file,O_RDWR|O_APPEND);
    uint64_t begin = milliSecond();
    for (int i = 0; i < test_times; ++i)
    {
        char prefix1 = i / 10 + '0';
        char prefix2 = i % 10 + '0';

        old_files[0] = prefix1;
        old_files[1] = prefix2;

        new_files[0] = prefix1;
        new_files[1] = prefix2;

        //int fd_new = open(new_files,O_WRONLY|O_APPEND);
        rename(old_files, new_files);
        //close(fd_new);
    }

    printf("%d times, use time %dms\n", test_times, milliSecond() - begin);
    
    return 0;
}