这几天我在GitHub上读了GIL的实现和python主循环的源码,总算对python的GIL有了大概的理解,现在来分享一下心得。
GIL源码上有这样一段注释:
The GIL is just a boolean variable (locked) whose access is protected by a mutex (gil_mutex), and whose changes are signalled by a condition variable (gil_cond). gil_mutex is taken for short periods of time, and therefore mostly uncontended.
这段时注释说明了GIL只是一个布尔值。不过为了控制各个线程对它的访问,设置了互斥锁和信号量。这几个变量被保存于结构体_gil_runtime_state之中,下面是结构体的定义:
struct _gil_runtime_state {
/* microseconds (the Python API uses seconds, though) */
unsigned long interval;
/* Last PyThreadState holding / having held the GIL. This helps us
know whether anyone else was scheduled after we dropped the GIL. */
_Py_atomic_address last_holder;
/* Whether the GIL is already taken (-1 if uninitialized). This is
atomic because it can be read without any lock taken in ceval.c. */
_Py_atomic_int locked;
/* Number of GIL switches since the beginning. */
unsigned long switch_number;
/* This condition variable allows one or several threads to wait
until the GIL is released. In addition, the mutex also protects
the above variables. */
PyCOND_T cond;
PyMUTEX_T mutex;
#ifdef FORCE_SWITCHING
/* This condition variable helps the GIL-releasing thread wait for
a GIL-awaiting thread to be scheduled and take the GIL. */
PyCOND_T switch_cond;
PyMUTEX_T switch_mutex;
#endif
};
这个结构体中还有两个重要变量一个是interval,是一个计时器,默认为5ms,当一个线程执行时间超过5ms,将进行线程切换;另一个是last_holder,用于记录当前运行线程的信息,当前线程准备释放GIL时,last_holder会更新为下一个即将运行的线程的信息,这样能保证当前线程释放GIL之后,其他某个等待的线程可以立刻执行。有个隐含的意思,GIL在任一时刻只允许一个线程执行。GIL源码上对这样做的原因也做出了解释:
This is meant to prohibit the latency-adverse behaviour on multi-core machines where one thread would speculatively release the GIL, but still run and end up being the first to re-acquire it, making the “timeslices” much longer than expected.
这种机制是为了防止出现一个线程刚刚释放GIL又重新获取,导致时间片分配不均,同时简化线程调度。由于一个解释进程在任一时刻只有一个线程在执行,所以在多核心CPU上运行的python多线程程序只会利用一个核心,完全没有做到并行,要实现并行只能使用多进程,然而多进程之间的通信又成了编程中的一大难题。所以python社区也有不少人提出过要废除GIL,然而结果并不好,运行效率不增反降。
接下来可以说说为什么GIL可以保证字节码层面上的线程安全了。首先我们看看python解释器是如何工作的:
for (;;) {
...
if (_Py_atomic_load_relaxed(
&_PyRuntime.ceval.gil_drop_request)) //判断是否有GIL释放请求,由interval决定
{
/* Give another thread a chance */
if (PyThreadState_Swap(NULL) != tstate) //这里进行线程切换,将当前线程置为NULL
Py_FatalError("ceval: tstate mix-up");
drop_gil(tstate); //释放GIL
/* Other threads may run now */
take_gil(tstate);
...
}
/* execute the instruction ... */
}
python解释器中有这样一个循环体,每一次循环都会执行一条解释器指令。进入循环后,先判断是否有释放GIL的请求(gil_drop_request=1)。若有,释放GIL,将CPU使用权交给别的线程;若无,继续执行下一条指令。也就是说,GIL的状态在解释器指令执行期间是不会发生变化的,这样就能保证当前线程不至于被其他线程干扰。
最后提醒一下,GIL只是保证解释器层面的线程安全,要保护源代码层面的对象,还需要python中锁的保护。