Releasing the GIL

What happens to our Python code when it calls a Rust function?
It waits for the Rust function to return:

 Time -->

          +------------+--------------------+------------+--------------------+
 Python:  |  Execute   | Call Rust Function |    Idle    |  Resume Execution  |
          +------------+--------------------+------------+--------------------+
                                 │                                ▲
                                 ▼                                │
          +------------+--------------------+------------+--------------------+
 Rust:    |    Idle    |       Idle         |  Execute   |  Return to Python  |
          +------------+--------------------+------------+--------------------+

The schema doesn't change even if the Rust function is multithreaded:

 Time -->

          +------------+--------------------+-------------------+--------------------+
 Python:  |  Execute   | Call Rust Function |       Idle        |  Resume Execution  |
          +------------+--------------------+-------------------+--------------------+
                                 │                                        ▲
                                 ▼                                        │
          +------------+--------------------+-------------------+--------------------+
 Rust:    |    Idle    |       Idle         | Execute Thread 1  |  Return to Python  |
          |            |                    | Execute Thread 2  |                    |
          +------------+--------------------+-------------------+--------------------+

It begs the question: can we have Python and Rust code running concurrently?
Yes! The focus point, once again, is the GIL.

Python access must be serialized

The GIL's job is to serialize all interactions with Python objects.
On the pyo3 side, this is modeled by the Python<'py> token: you can only get an instance of Python<'py> if you're holding the GIL. Going further, you can only interact with Python objects via smart pointers like Borrowed<'py, T> or Owned<'py, T>, which internally hold a Python<'py> instance.
There's no way around it: any interaction with Python objects must be serialized. But, here's the kicker: not all Rust code needs to interact with Python objects!

Python::allow_threads

For example, consider a Rust function that calculates the nth Fibonacci number:

#![allow(unused)]
fn main() {
#[pyfunction]
fn fibonacci(n: u64) -> u64 {
    let mut a = 0;
    let mut b = 1;
    for _ in 0..n {
        let tmp = a;
        a = b;
        b = tmp + b;
    }
    a
}
}

There's no Python object in sight! We're just offloading a computation to Rust.
In principle, we could spawn a thread to run this function while the main thread continues executing Python code:

from threading import Thread

def other_work():
    print("I'm doing other work!")

t = Thread(target=fibonacci, args=(10,))
t.start()
other_work()
t.join()

As it stands, other_work and fibonacci will not be run in parallel: our fibonacci routine is still holding the GIL, even though it doesn't need it.
We can fix it by explicitly releasing the GIL:

#![allow(unused)]
fn main() {
#[pyfunction]
fn fibonacci(py: Python<'_>, n: u64) -> u64 {
    py.allow_threads(|| {
        let mut a = 0;
        let mut b = 1;
        for _ in 0..n {
            let tmp = a;
            a = b;
            b = tmp + b;
        }
        a
    })
}
}

Python::allow_threads releases the GIL while executing the closure passed to it.
This frees up the Python interpreter to run other Python code, such as the other_work function in our example, while the Rust thread is busy calculating the nth Fibonacci number.

Using the same line diagram as before, we have the following:

 Time -->

          +------------+--------------------+-------------------+--------------------+
 Python:  |  Execute   | Call Rust Function |    other_work()   |      t.join()      |
          +------------+--------------------+-------------------+--------------------+
                                 │                                        ▲
                                 ▼                                        │
          +------------+--------------------+-------------------+--------------------+
 Rust:    |    Idle    |       Idle         |    fibonacci(n)   |  Return to Python  |
          +------------+--------------------+-------------------+--------------------+
                                                     ▲
                                                     │
                                            Python and Rust code
                                          running concurrently here

Ungil

Python::allow_threads is only sound if the closure doesn't interact with Python objects.
If that's not the case, we end up with undefined behavior: Rust code touching Python objects while the Python interpreter is running other Python code, assuming nothing else is happening to those objects thanks to the GIL. A recipe for disaster!

It'd be ideal to rely on the type system to enforce this constraint for us at compile-time, in true Rust fashion—"if it compiles, it's safe."
pyo3 tries to follow this principle with the Ungil marker trait: only types that are safe to access without the GIL can implement Ungil. The trait is then used to constrain the arguments of Python::allow_threads:

#![allow(unused)]
fn main() {
pub fn allow_threads<T, F>(self, f: F) -> T
where
    F: Ungil + FnOnce() -> T,
    T: Ungil,
{
    // ...
}
}

Unfortunately, Ungil is not perfect. On stable Rust, it leans on the Send trait, but that allows for some unsafe interactions with Python objects. The tracking is more precise on nightly Rust¹, but it doesn't catch every possible misuse of Python::allow_threads.

My recommendation: if you're using Python::allow_threads, trigger an additional run of your CI pipeline using the nightly Rust compiler to catch more issues. On top of that, review your code carefully.

Exercise

The exercise for this section is located in 03_concurrency/03_releasing_the_gil