Skip to content

Commit

Permalink
rename GIL.release to GIL.unlock and use lock/unlock terminology cons…
Browse files Browse the repository at this point in the history
…istently
  • Loading branch information
Christopher Doris committed Aug 3, 2024
1 parent 9dbf65e commit 131c312
Show file tree
Hide file tree
Showing 10 changed files with 52 additions and 52 deletions.
12 changes: 6 additions & 6 deletions docs/src/juliacall.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,13 +132,13 @@ caveats.

Most importantly, you can only call Python code while Python's
[Global Interpreter Lock (GIL)](https://docs.python.org/3/glossary.html#term-global-interpreter-lock)
is held by the current thread. You can use JuliaCall from any Python thread, and the GIL
will be held whenever any JuliaCall function is used. However, to leverage the benefits
of multi-threading, you can release the GIL while executing any Julia code that does not
is locked by the current thread. You can use JuliaCall from any Python thread, and the GIL
will be locked whenever any JuliaCall function is used. However, to leverage the benefits
of multi-threading, you can unlock the GIL while executing any Julia code that does not
interact with Python.

The simplest way to do this is using the `_jl_call_nogil` method on Julia functions to
call the function with the GIL released.
call the function with the GIL unlocked.

```python
from concurrent.futures import ThreadPoolExecutor, wait
Expand All @@ -149,14 +149,14 @@ wait(fs)
```

In the above example, we call `Libc.systemsleep(5)` on four threads. Because we
called it with `_jl_call_nogil`, the GIL was released, allowing the threads to run in
called it with `_jl_call_nogil`, the GIL was unlocked, allowing the threads to run in
parallel, taking about 5 seconds in total.

If we did not use `_jl_call_nogil` (i.e. if we did `pool.submit(jl.Libc.systemsleep, 5)`)
then the above code will take 20 seconds because the sleeps run one after another.

It is very important that any function called with `_jl_call_nogil` does not interact
with Python at all unless it re-acquires the GIL first, such as by using
with Python at all unless it re-locks the GIL first, such as by using
[PythonCall.GIL.@lock](@ref).

You can also use [multi-threading from Julia](@ref jl-multi-threading).
Expand Down
4 changes: 2 additions & 2 deletions docs/src/pythoncall-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -226,8 +226,8 @@ See also [`juliacall.AnyValue._jl_call_nogil`](@ref julia-wrappers).
```@docs
PythonCall.GIL.lock
PythonCall.GIL.@lock
PythonCall.GIL.release
PythonCall.GIL.@release
PythonCall.GIL.unlock
PythonCall.GIL.@unlock
PythonCall.GC.gc
```

Expand Down
20 changes: 10 additions & 10 deletions docs/src/pythoncall.md
Original file line number Diff line number Diff line change
Expand Up @@ -370,16 +370,16 @@ caveats.

Most importantly, you can only call Python code while Python's
[Global Interpreter Lock (GIL)](https://docs.python.org/3/glossary.html#term-global-interpreter-lock)
is held by the current thread. Ordinarily, the GIL is held by the main thread in Julia,
so if you want to run Python code on any other thread, you must release the GIL from the
main thread and then re-acquire it while running any Python code on other threads.
is locked by the current thread. Ordinarily, the GIL is locked by the main thread in Julia,
so if you want to run Python code on any other thread, you must unlock the GIL from the
main thread and then re-lock it while running any Python code on other threads.

This is made possible by the macros [`PythonCall.GIL.@release`](@ref) and
[`PythonCall.GIL.@lock`](@ref) or the functions [`PythonCall.GIL.release`](@ref) and
This is made possible by the macros [`PythonCall.GIL.@unlock`](@ref) and
[`PythonCall.GIL.@lock`](@ref) or the functions [`PythonCall.GIL.unlock`](@ref) and
[`PythonCall.GIL.lock`](@ref) with this pattern:

```julia
PythonCall.GIL.@release Threads.@threads for i in 1:4
PythonCall.GIL.@unlock Threads.@threads for i in 1:4
PythonCall.GIL.@lock pyimport("time").sleep(5)
end
```
Expand All @@ -388,17 +388,17 @@ In the above example, we call `time.sleep(5)` four times in parallel. If Julia w
started with at least four threads (`julia -t4`) then the above code will take about
5 seconds.

Both `@release` and `@lock` are important. If the GIL were not released, then a deadlock
Both `@unlock` and `@lock` are important. If the GIL were not unlocked, then a deadlock
would occur when attempting to lock the already-locked GIL from the threads. If the GIL
were not re-acquired, then Python would crash when interacting with it.
were not re-locked, then Python would crash when interacting with it.

You can also use [multi-threading from Python](@ref py-multi-threading).

### Caveat: Garbage collection

If Julia's GC collects any Python objects from a thread where the GIL is not currently
held, then those Python objects will not immediately be deleted. Instead they will be
locked, then those Python objects will not immediately be deleted. Instead they will be
queued to be deleted in a later GC pass.

If you find you have many Python objects not being deleted, you can call
[`PythonCall.GC.gc()`](@ref) or `GC.gc()` while the GIL is held to clear the queue.
[`PythonCall.GC.gc()`](@ref) or `GC.gc()` while the GIL is locked to clear the queue.
4 changes: 2 additions & 2 deletions docs/src/releasenotes.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,11 @@
* `GC.disable()` and `GC.enable()` are now a no-op and deprecated since they are no
longer required for thread-safety. These will be removed in v1.
* Adds `GC.gc()`.
* Adds module `GIL` with `lock()`, `release()`, `@lock` and `@release` for handling the
* Adds module `GIL` with `lock()`, `unlock()`, `@lock` and `@unlock` for handling the
Python Global Interpreter Lock. In combination with the above improvements, these
allow Julia and Python to co-operate on multiple threads.
* Adds method `_jl_call_nogil` to `juliacall.AnyValue` and `juliacall.RawValue` to call
Julia functions with the GIL released.
Julia functions with the GIL unlocked.

## 0.9.21 (2024-07-20)
* `Serialization.serialize` can use `dill` instead of `pickle` by setting the env var `JULIA_PYTHONCALL_PICKLE=dill`.
Expand Down
2 changes: 1 addition & 1 deletion pytest/test_all.py
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ def test_call_nogil(yld, raw):
from time import time
from juliacall import Main as jl

# julia implementation of sleep which releases the GIL
# julia implementation of sleep which unlocks the GIL
if yld:
# use sleep, which yields
jsleep = jl.sleep
Expand Down
34 changes: 17 additions & 17 deletions src/GIL/GIL.jl
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
Handling the Python Global Interpreter Lock.
See [`lock`](@ref), [`@lock`](@ref), [`release`](@ref) and [`@release`](@ref).
See [`lock`](@ref), [`@lock`](@ref), [`unlock`](@ref) and [`@unlock`](@ref).
"""
module GIL

Expand All @@ -12,11 +12,11 @@ using ..C: C
"""
lock(f)
Acquire the GIL, compute `f()`, release the GIL, then return the result of `f()`.
Unlock the GIL, compute `f()`, unlock the GIL, then return the result of `f()`.
Use this to run Python code from threads that do not currently hold the GIL, such as new
threads. Since the main Julia thread holds the GIL by default, you will need to
[`release`](@ref) the GIL before using this function.
[`unlock`](@ref) the GIL before using this function.
See [`@lock`](@ref) for the macro form.
"""
Expand All @@ -32,11 +32,11 @@ end
"""
@lock expr
Acquire the GIL, compute `expr`, release the GIL, then return the result of `expr`.
Unlock the GIL, compute `expr`, unlock the GIL, then return the result of `expr`.
Use this to run Python code from threads that do not currently hold the GIL, such as new
threads. Since the main Julia thread holds the GIL by default, you will need to
[`@release`](@ref) the GIL before using this function.
[`@unlock`](@ref) the GIL before using this function.
The macro equivalent of [`lock`](@ref).
"""
Expand All @@ -52,17 +52,17 @@ macro lock(expr)
end

"""
release(f)
unlock(f)
Release the GIL, compute `f()`, re-acquire the GIL, then return the result of `f()`.
Unlock the GIL, compute `f()`, re-lock the GIL, then return the result of `f()`.
Use this to run non-Python code with the GIL released, so allowing another thread to run
Python code. That other thread can be a Julia thread, which must acquire the GIL using
Use this to run non-Python code with the GIL unlocked, so allowing another thread to run
Python code. That other thread can be a Julia thread, which must lock the GIL using
[`lock`](@ref).
See [`@release`](@ref) for the macro form.
See [`@unlock`](@ref) for the macro form.
"""
function release(f)
function unlock(f)
state = C.PyEval_SaveThread()
try
f()
Expand All @@ -72,17 +72,17 @@ function release(f)
end

"""
@release expr
@unlock expr
Release the GIL, compute `expr`, re-acquire the GIL, then return the result of `expr`.
Unlock the GIL, compute `expr`, re-lock the GIL, then return the result of `expr`.
Use this to run non-Python code with the GIL released, so allowing another thread to run
Python code. That other thread can be a Julia thread, which must acquire the GIL using
Use this to run non-Python code with the GIL unlocked, so allowing another thread to run
Python code. That other thread can be a Julia thread, which must lock the GIL using
[`@lock`](@ref).
The macro equivalent of [`release`](@ref).
The macro equivalent of [`unlock`](@ref).
"""
macro release(expr)
macro unlock(expr)
quote
state = C.PyEval_SaveThread()
try
Expand Down
6 changes: 3 additions & 3 deletions src/JlWrap/any.jl
Original file line number Diff line number Diff line change
Expand Up @@ -55,12 +55,12 @@ function pyjlany_call_nogil(self, args_::Py, kwargs_::Py)
if pylen(kwargs_) > 0
args = pyconvert(Vector{Any}, args_)
kwargs = pyconvert(Dict{Symbol,Any}, kwargs_)
ans = Py(GIL.@release self(args...; kwargs...))
ans = Py(GIL.@unlock self(args...; kwargs...))
elseif pylen(args_) > 0
args = pyconvert(Vector{Any}, args_)
ans = Py(GIL.@release self(args...))
ans = Py(GIL.@unlock self(args...))
else
ans = Py(GIL.@release self())
ans = Py(GIL.@unlock self())
end
pydel!(args_)
pydel!(kwargs_)
Expand Down
6 changes: 3 additions & 3 deletions src/JlWrap/raw.jl
Original file line number Diff line number Diff line change
Expand Up @@ -44,12 +44,12 @@ function pyjlraw_call_nogil(self, args_::Py, kwargs_::Py)
if pylen(kwargs_) > 0
args = pyconvert(Vector{Any}, args_)
kwargs = pyconvert(Dict{Symbol,Any}, kwargs_)
ans = pyjlraw(GIL.@release self(args...; kwargs...))
ans = pyjlraw(GIL.@unlock self(args...; kwargs...))
elseif pylen(args_) > 0
args = pyconvert(Vector{Any}, args_)
ans = pyjlraw(GIL.@release self(args...))
ans = pyjlraw(GIL.@unlock self(args...))
else
ans = pyjlraw(GIL.@release self())
ans = pyjlraw(GIL.@unlock self())
end
pydel!(args_)
pydel!(kwargs_)
Expand Down
4 changes: 2 additions & 2 deletions test/GC.jl
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
@testitem "GC.gc()" begin
let
pyobjs = map(pylist, 1:100)
PythonCall.GIL.@release Threads.@threads for obj in pyobjs
PythonCall.GIL.@unlock Threads.@threads for obj in pyobjs
finalize(obj)
end
end
Expand All @@ -13,7 +13,7 @@ end
@testitem "GC.GCHook" begin
let
pyobjs = map(pylist, 1:100)
PythonCall.GIL.@release Threads.@threads for obj in pyobjs
PythonCall.GIL.@unlock Threads.@threads for obj in pyobjs
finalize(obj)
end
end
Expand Down
12 changes: 6 additions & 6 deletions test/GIL.jl
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
@testitem "release and lock" begin
# This calls Python's time.sleep(1) twice concurrently. Since sleep() releases the
@testitem "unlock and lock" begin
# This calls Python's time.sleep(1) twice concurrently. Since sleep() unlocks the
# GIL, these can happen in parallel if Julia has at least 2 threads.
function threaded_sleep()
PythonCall.GIL.release() do
PythonCall.GIL.unlock() do
Threads.@threads for i = 1:2
PythonCall.GIL.lock() do
pyimport("time").sleep(1)
Expand All @@ -20,11 +20,11 @@
end
end

@testitem "@release and @lock" begin
# This calls Python's time.sleep(1) twice concurrently. Since sleep() releases the
@testitem "@unlock and @lock" begin
# This calls Python's time.sleep(1) twice concurrently. Since sleep() unlocks the
# GIL, these can happen in parallel if Julia has at least 2 threads.
function threaded_sleep()
PythonCall.GIL.@release Threads.@threads for i = 1:2
PythonCall.GIL.@unlock Threads.@threads for i = 1:2
PythonCall.GIL.@lock pyimport("time").sleep(1)
end
end
Expand Down

0 comments on commit 131c312

Please sign in to comment.