A Detailed Explanation of Python GIL

What is GIL

GIL (Global Interpreter Lock) is a global lock in the CPython interpreter:

At any given time, only one thread can execute Python bytecode;
Even if you have many threads in the same process, only one thread can actually run Python code (the others are waiting for the lock).

It is important to note that Python has multiple implementations, such as:CPython, PyPy, Jython, etc. What we usually refer to as Python actually means CPython.

To clarify: they are all different implementations of Python. Python is a language (syntax/specification), and CPython, PyPy, Jython, and IronPython are different implementations of this language, similar to:

Java has different virtual machines like HotSpot, OpenJ9, etc.
JavaScript has different engines like V8, SpiderMonkey, JavaScriptCore, etc.

As long as they conform to the Python language specification, they can all be called “Python implementations”.

CPython: The “default Python” you are using now, downloaded from the official website python.org, and when you type python in the command line, it is most likely CPython.

Characteristics:

It is an interpreter written in C, hence called C-Python
This is the official/standard implementation with the best ecosystem
It has GIL (Global Interpreter Lock)
Most extension libraries (like numpy, pandas, django, ortools) are designed for CPython

Advantages:

Stable, with the strongest compatibility and the most libraries
Documentation, community, and tutorials are almost all based on CPython.

Disadvantages:

GIL → pure Python multithreading cannot truly utilize multiple cores for CPU-intensive tasks
Performance is not particularly extreme (but sufficient + can be supplemented with C extensions)

Almost everyone defaults to using CPython, and so do you.

GIL is a mechanism at the implementation level of CPython, not a syntactic feature of the Python language itself; PyPy, Jython, IronPython, and other implementations can have different strategies (some do not have the traditional GIL or use other mechanisms).

Why was GIL introduced?

The early design goal was mainly to simplify the implementation of the interpreter under the multithreading + reference counting memory management model, reducing the complexity of concurrent bugs (especially in memory management).

CPython uses reference counting to manage the lifecycle of objects, with typical operations:

Py_INCREF(obj);
Py_DECREF(obj);

These operations can occur simultaneously in a multithreaded environment. If there is no unified lock to protect the reference counts of these shared objects:

There may be reference count write conflicts;
This can lead to objects being incorrectly released or never released;
Debugging such memory errors is very painful.

The core purpose of GIL:

To make CPython‘s memory management (especially reference counting) simple and safe in a multithreaded environment;
To avoid adding various fine-grained locks on all objects and internal structures (which would make implementation extremely complex, performance unpredictable, and prone to deadlocks).

In summary: GIL is a brute-force simple solution that “wraps the interpreter with a big lock,” trading engineering complexity for simplicity and safety in implementation.

What problems does GIL solve?

✅ Avoids competition for object reference counting under multithreading;
✅ Reduces the complexity of CPython implementation (fewer locks, less fine-grained synchronization);
✅ Allows C extension modules (while holding GIL) to assume they are “thread-safe” by default (without needing to manage concurrent access to Python layer objects).

Advantages and Disadvantages of GIL

Advantages:

2.1 Simple and safe implementation

2.2 Friendly to I/O intensive tasks

2.3 C extension libraries can bypass GIL for multicore computing

Disadvantages: (points of criticism)

2.4 CPU-intensive Python code cannot truly utilize multiple cores

2.5 Thread semantics are inconsistent with most languages

2.6 Increases the understanding cost for performance tuning

2.7 Makes Python unsuitable as a high-performance computing kernel (pure Python implementation)

Implement core algorithms in C/C++/Rust;
Or use multiprocessing for multiple processes;
Or use distributed computing frameworks (Ray, Dask, Spark, etc.).

To truly perform large-scale parallel computing, you usually need to:

Differentiate between CPU-intensive vs I/O-intensive;
Differentiate between Python layer code vs C/extension layer code;

You need to distinguish:
Threads behave completely differently in different scenarios.

In other languages (like Java), “multithreading == can utilize multicore for parallel execution”;
In CPython, “multithreading == convenient concurrency model, but does not equate to multicore parallelism”.

When multiple threads run Python computation logic in the same process, they still “take turns acquiring GIL”;
Multithreading may even be slower (context switching + GIL contention).

For example, libraries like numpy, ortools, etc., use C/C++ code internally to create threads and release GIL;
Although there is GIL at the Python thread level, the actual computation occurs at the C level, which can fully utilize multiple cores.

I/O operations (like network, disk) usually release GIL during system calls;
That is: when a thread is waiting for I/O, other threads can acquire GIL to execute;
Therefore, multithreading can still achieve good concurrent performance in I/O-bound scenarios (like web scraping, network service clients, etc.).

With only one global lock, many data structures within the interpreter can assume “will not be accessed concurrently”;
Memory management (reference counting) has low implementation costs.

Impact of GIL on Python Threads and Coroutines

Impact on Threads (threading)

Core: Multiple threads cannot execute pure Python computations (CPU-bound) in parallel on multiple cores.

Specific performance:

CPU-intensive: use multiprocessing or C extensions to bypass GIL; I/O-intensive: multithreading is very suitable.

For example, multiple threads performing large loops, calculating prime numbers, compressing, encrypting, etc., using pure Python operations;
They will constantly compete for GIL, and when executing, “one thread runs for a while, switches, then another thread runs for a while,” but at any moment only one thread is executing Python code;
Multithreading not only cannot speed up, but may slow down.

For example, opening 100 threads to send HTTP requests;
Most of the time, requests are “waiting for the network”; during this time, threads will release GIL during I/O calls;
The overall concurrent effect is good, and multithreading is valuable.

I/O-intensive tasks:
CPU-intensive tasks:

Impact on Coroutines (asyncio, async/await)

Coroutines are a “concurrent scheduling mechanism within a single thread”:

But the goal of coroutines is not parallelism across multiple cores, but:

So:

For I/O-intensive tasks: coroutines + single-threaded are completely feasible and efficient (without creating a large number of threads);
For CPU-intensive tasks: coroutines also cannot run multiple tasks simultaneously; GIL and threads will limit it.

It essentially packs a lot of I/O wait time “into one thread”;
Avoids the scheduling and memory overhead of creating a thread for each I/O task;

It is essentially one thread + an event loop;
By using await to actively “yield execution rights,” allowing other coroutines to have a chance to run;
However, it still executes within a single OS thread, so: coroutines do not bypass GIL and are still constrained by it.

How to Bypass GIL?

The Python community has debated for many years about whether to keep GIL. Although there have been attempts to create no-GIL implementations in recent years, they also bring new problems. Overall, there has been progress, but it is not something that can be completely replaced in the short term.

Strictly speaking, GIL cannot be bypassed within a single process, but we can consider the problem from a different perspective: since it cannot be bypassed within a process, why not start multiple Python processes to solve it? Yes, this is actually how it is solved, for example:multiprocessing and gunicorn.

What is GIL

Why was GIL introduced?

Impact of GIL on Python Threads and Coroutines

How to Bypass GIL?

Related posts

Leave a Comment Cancel reply