文件锁filelock源码解读

---

前言

笔者遇到需要在不同的进程中访问同一个文件的场景，但是这个文件是临界资源，不能同时访问，因此需要利用锁机制来控制不同进程之间的访问。于是，引出了进程锁（即文件锁）这个概念。经过查阅资料，笔者发现filelock库可以满足我们的要求。笔者在使用之余也想看看到底是怎么实现的文件锁，于是便对源码（3.2.1版本）做了解析。

---

一、filelock是什么？

filelock 是一种进程锁，可以控制不同进程之间对临界资源的访问。类似于普通的线程锁，会在全局中初始化一把线程锁，然后在不同现成中去加锁和释放锁。filelock的核心在于设置这么一把进程锁，在不同进程中加锁和释放锁。
代码如下（示例）：

import filelock

lock = filelock.FileLock("./my.lock")

try:
    lock.acquire()
    
    # 访问其他进程也会访问的临界资源，比如文件
    
    lock.release()
except TimeoutError:
    pass
finally:
    if lock.is_locked:
        lock.release()

二、核心类

1.BaseFileLock

BaseFileLock类是其他所有类的基础类，也是核心类，这个类实现了类似threading.Lock类的功能(acquire, release)，使得文件锁的使用形式和threading.Lock类基本一致。WindowsFileLock类和UnixFileLock类则是具体实现应该如何锁定一个文件，让其他进程无法锁定文件，很明显，进程都在操作系统运行，这依赖操作系统的命令。先看BaseFileLock类：
代码如下（示例）：

    def acquire(self, timeout=None, poll_intervall=0.05):
    	# Use the default timeout, if no timeout is provided.
        if timeout is None:
            timeout = self.timeout

        # Increment the number right at the beginning. We can still undo it, if something fails.
        with self._thread_lock:
            self._lock_counter += 1

        lock_id = id(self)
        lock_filename = self._lock_file
        start_time = time.time()
        try:
            while True:
                with self._thread_lock:
                    if not self.is_locked:
                        _LOGGER.debug("Attempting to acquire lock %s on %s", lock_id, lock_filename)
                        self._acquire()

                if self.is_locked:
                    _LOGGER.debug("Lock %s acquired on %s", lock_id, lock_filename)
                    break
                elif 0 <= timeout < time.time() - start_time:
                    _LOGGER.debug("Timeout on acquiring lock %s on %s", lock_id, lock_filename)
                    raise Timeout(self._lock_file)
                else:
                    msg = "Lock %s not acquired on %s, waiting %s seconds ..."
                    _LOGGER.debug(msg, lock_id, lock_filename, poll_intervall)
                    time.sleep(poll_intervall)
        except BaseException:  # Something did go wrong, so decrement the counter.
            with self._thread_lock:
                self._lock_counter = max(0, self._lock_counter - 1)
            raise
        return AcquireReturnProxy(lock=self)
        
    def release(self, force=False):
        """
        Releases the file lock. Please note, that the lock is only completely released, if the lock counter is 0. Also
        note, that the lock file itself is not automatically deleted.

        :param force: If true, the lock counter is ignored and the lock is released in every case/
        """
        with self._thread_lock:

            if self.is_locked:
                self._lock_counter -= 1

                if self._lock_counter == 0 or force:
                    lock_id, lock_filename = id(self), self._lock_file

                    _LOGGER.debug("Attempting to release lock %s on %s", lock_id, lock_filename)
                    self._release()
                    self._lock_counter = 0
                    _LOGGER.debug("Lock %s released on %s", lock_id, lock_filename)

根据上述代码可知，就是在死循环中调用真正的_aquire()方法去锁定文件，如果设置超时时间，超时就会退出。而release()方法就更简单了，判断如果被锁定了就释放锁。

2.WindowsFileLock

WindowsFileLock类就是BaseFileLock的子类，实现了BaseFileLock的_aquire和_release方法，这里面就是在window平台下如何锁定一个文件，实际利用的是msvcrt库
代码如下（示例）：

class WindowsFileLock(BaseFileLock):
    """Uses the :func:`msvcrt.locking` function to hard lock the lock file on windows systems."""

    def _acquire(self):
        raise_on_exist_ro_file(self._lock_file)
        mode = (
            os.O_RDWR  # open for read and write
            | os.O_CREAT  # create file if not exists
            | os.O_TRUNC  # truncate file  if not empty
        )
        try:
            fd = os.open(self._lock_file, mode)
        except OSError as exception:
            if exception.errno == ENOENT:  # No such file or directory
                raise
        else:
            try:
                msvcrt.locking(fd, msvcrt.LK_NBLCK, 1)
            except (OSError, IOError):  # noqa: B014 # IOError is not OSError on python 2
                os.close(fd)
            else:
                self._lock_file_fd = fd

    def _release(self):
        fd = self._lock_file_fd
        self._lock_file_fd = None
        msvcrt.locking(fd, msvcrt.LK_UNLCK, 1)
        os.close(fd)

        try:
            os.remove(self._lock_file)
        # Probably another instance of the application hat acquired the file lock.
        except OSError:
            pass

3.UnixFileLock

和WindowsFileLock类似，不同之处在于unix系统利用fcntl库来锁定文件，此处就不再赘述。

4.SoftFileLock

SoftFileLock 就更加简单了，直接利用os.O_EXCL（互斥）的模式创建文件，在文件创建后，python代码中无法创建该文件，同样达到了锁定的效果，然后释放的时候删除文件相当于解除锁定。但是，这种方法会导致如果程序崩溃，残留锁定文件后导致死锁。

class SoftFileLock(BaseFileLock):
    """Simply watches the existence of the lock file."""

    def _acquire(self):
        raise_on_exist_ro_file(self._lock_file)
        # first check for exists and read-only mode as the open will mask this case as EEXIST
        mode = (
            os.O_WRONLY  # open for writing only
            | os.O_CREAT
            | os.O_EXCL  # together with above raise EEXIST if the file specified by filename exists
            | os.O_TRUNC  # truncate the file to zero byte
        )
        try:
            fd = os.open(self._lock_file, mode)
        except OSError as exception:
            if exception.errno == EEXIST:  # expected if cannot lock
                pass
            elif exception.errno == ENOENT:  # No such file or directory - parent directory is missing
                raise
            elif exception.errno == EACCES and sys.platform != "win32":  # Permission denied - parent dir is R/O
                raise  # note windows does not allow you to make a folder r/o only files
        else:
            self._lock_file_fd = fd

    def _release(self):
        os.close(self._lock_file_fd)
        self._lock_file_fd = None
        try:
            os.remove(self._lock_file)
        except OSError:  # the file is already deleted and that's what we want
            pass

5.FileLock

Filelock就是上述三个类的别名，是windows系统就用WindowsFileLock，有fcntl库就用UnixFileLock类，否则就是SoftFileLock。

---

总结

以上就是今天要讲的内容，本文介绍了filelock中的几个核心类，经过粗略的了解，读者应该对如何使用filelock类有几分心得了。