diff --git a/docs/src/juliacall.md b/docs/src/juliacall.md index 9d46ac57..2587aa10 100644 --- a/docs/src/juliacall.md +++ b/docs/src/juliacall.md @@ -132,13 +132,13 @@ caveats. Most importantly, you can only call Python code while Python's [Global Interpreter Lock (GIL)](https://docs.python.org/3/glossary.html#term-global-interpreter-lock) -is held by the current thread. You can use JuliaCall from any Python thread, and the GIL -will be held whenever any JuliaCall function is used. However, to leverage the benefits -of multi-threading, you can release the GIL while executing any Julia code that does not +is locked by the current thread. You can use JuliaCall from any Python thread, and the GIL +will be locked whenever any JuliaCall function is used. However, to leverage the benefits +of multi-threading, you can unlock the GIL while executing any Julia code that does not interact with Python. The simplest way to do this is using the `_jl_call_nogil` method on Julia functions to -call the function with the GIL released. +call the function with the GIL unlocked. ```python from concurrent.futures import ThreadPoolExecutor, wait @@ -149,14 +149,14 @@ wait(fs) ``` In the above example, we call `Libc.systemsleep(5)` on four threads. Because we -called it with `_jl_call_nogil`, the GIL was released, allowing the threads to run in +called it with `_jl_call_nogil`, the GIL was unlocked, allowing the threads to run in parallel, taking about 5 seconds in total. If we did not use `_jl_call_nogil` (i.e. if we did `pool.submit(jl.Libc.systemsleep, 5)`) then the above code will take 20 seconds because the sleeps run one after another. It is very important that any function called with `_jl_call_nogil` does not interact -with Python at all unless it re-acquires the GIL first, such as by using +with Python at all unless it re-locks the GIL first, such as by using [PythonCall.GIL.@lock](@ref). You can also use [multi-threading from Julia](@ref jl-multi-threading). diff --git a/docs/src/pythoncall-reference.md b/docs/src/pythoncall-reference.md index 018cfdfe..b1dd795e 100644 --- a/docs/src/pythoncall-reference.md +++ b/docs/src/pythoncall-reference.md @@ -226,8 +226,8 @@ See also [`juliacall.AnyValue._jl_call_nogil`](@ref julia-wrappers). ```@docs PythonCall.GIL.lock PythonCall.GIL.@lock -PythonCall.GIL.release -PythonCall.GIL.@release +PythonCall.GIL.unlock +PythonCall.GIL.@unlock PythonCall.GC.gc ``` diff --git a/docs/src/pythoncall.md b/docs/src/pythoncall.md index a7767d42..8b8f19a1 100644 --- a/docs/src/pythoncall.md +++ b/docs/src/pythoncall.md @@ -370,16 +370,16 @@ caveats. Most importantly, you can only call Python code while Python's [Global Interpreter Lock (GIL)](https://docs.python.org/3/glossary.html#term-global-interpreter-lock) -is held by the current thread. Ordinarily, the GIL is held by the main thread in Julia, -so if you want to run Python code on any other thread, you must release the GIL from the -main thread and then re-acquire it while running any Python code on other threads. +is locked by the current thread. Ordinarily, the GIL is locked by the main thread in Julia, +so if you want to run Python code on any other thread, you must unlock the GIL from the +main thread and then re-lock it while running any Python code on other threads. -This is made possible by the macros [`PythonCall.GIL.@release`](@ref) and -[`PythonCall.GIL.@lock`](@ref) or the functions [`PythonCall.GIL.release`](@ref) and +This is made possible by the macros [`PythonCall.GIL.@unlock`](@ref) and +[`PythonCall.GIL.@lock`](@ref) or the functions [`PythonCall.GIL.unlock`](@ref) and [`PythonCall.GIL.lock`](@ref) with this pattern: ```julia -PythonCall.GIL.@release Threads.@threads for i in 1:4 +PythonCall.GIL.@unlock Threads.@threads for i in 1:4 PythonCall.GIL.@lock pyimport("time").sleep(5) end ``` @@ -388,17 +388,17 @@ In the above example, we call `time.sleep(5)` four times in parallel. If Julia w started with at least four threads (`julia -t4`) then the above code will take about 5 seconds. -Both `@release` and `@lock` are important. If the GIL were not released, then a deadlock +Both `@unlock` and `@lock` are important. If the GIL were not unlocked, then a deadlock would occur when attempting to lock the already-locked GIL from the threads. If the GIL -were not re-acquired, then Python would crash when interacting with it. +were not re-locked, then Python would crash when interacting with it. You can also use [multi-threading from Python](@ref py-multi-threading). ### Caveat: Garbage collection If Julia's GC collects any Python objects from a thread where the GIL is not currently -held, then those Python objects will not immediately be deleted. Instead they will be +locked, then those Python objects will not immediately be deleted. Instead they will be queued to be deleted in a later GC pass. If you find you have many Python objects not being deleted, you can call -[`PythonCall.GC.gc()`](@ref) or `GC.gc()` while the GIL is held to clear the queue. +[`PythonCall.GC.gc()`](@ref) or `GC.gc()` while the GIL is locked to clear the queue. diff --git a/docs/src/releasenotes.md b/docs/src/releasenotes.md index f5d5eaef..5ce68c15 100644 --- a/docs/src/releasenotes.md +++ b/docs/src/releasenotes.md @@ -7,11 +7,11 @@ * `GC.disable()` and `GC.enable()` are now a no-op and deprecated since they are no longer required for thread-safety. These will be removed in v1. * Adds `GC.gc()`. -* Adds module `GIL` with `lock()`, `release()`, `@lock` and `@release` for handling the +* Adds module `GIL` with `lock()`, `unlock()`, `@lock` and `@unlock` for handling the Python Global Interpreter Lock. In combination with the above improvements, these allow Julia and Python to co-operate on multiple threads. * Adds method `_jl_call_nogil` to `juliacall.AnyValue` and `juliacall.RawValue` to call - Julia functions with the GIL released. + Julia functions with the GIL unlocked. ## 0.9.21 (2024-07-20) * `Serialization.serialize` can use `dill` instead of `pickle` by setting the env var `JULIA_PYTHONCALL_PICKLE=dill`. diff --git a/pytest/test_all.py b/pytest/test_all.py index c7adac5b..a895398a 100644 --- a/pytest/test_all.py +++ b/pytest/test_all.py @@ -125,7 +125,7 @@ def test_call_nogil(yld, raw): from time import time from juliacall import Main as jl - # julia implementation of sleep which releases the GIL + # julia implementation of sleep which unlocks the GIL if yld: # use sleep, which yields jsleep = jl.sleep diff --git a/src/GIL/GIL.jl b/src/GIL/GIL.jl index c1e89f23..fb5730bd 100644 --- a/src/GIL/GIL.jl +++ b/src/GIL/GIL.jl @@ -3,7 +3,7 @@ Handling the Python Global Interpreter Lock. -See [`lock`](@ref), [`@lock`](@ref), [`release`](@ref) and [`@release`](@ref). +See [`lock`](@ref), [`@lock`](@ref), [`unlock`](@ref) and [`@unlock`](@ref). """ module GIL @@ -12,11 +12,11 @@ using ..C: C """ lock(f) -Acquire the GIL, compute `f()`, release the GIL, then return the result of `f()`. +Unlock the GIL, compute `f()`, unlock the GIL, then return the result of `f()`. Use this to run Python code from threads that do not currently hold the GIL, such as new threads. Since the main Julia thread holds the GIL by default, you will need to -[`release`](@ref) the GIL before using this function. +[`unlock`](@ref) the GIL before using this function. See [`@lock`](@ref) for the macro form. """ @@ -32,11 +32,11 @@ end """ @lock expr -Acquire the GIL, compute `expr`, release the GIL, then return the result of `expr`. +Unlock the GIL, compute `expr`, unlock the GIL, then return the result of `expr`. Use this to run Python code from threads that do not currently hold the GIL, such as new threads. Since the main Julia thread holds the GIL by default, you will need to -[`@release`](@ref) the GIL before using this function. +[`@unlock`](@ref) the GIL before using this function. The macro equivalent of [`lock`](@ref). """ @@ -52,17 +52,17 @@ macro lock(expr) end """ - release(f) + unlock(f) -Release the GIL, compute `f()`, re-acquire the GIL, then return the result of `f()`. +Unlock the GIL, compute `f()`, re-lock the GIL, then return the result of `f()`. -Use this to run non-Python code with the GIL released, so allowing another thread to run -Python code. That other thread can be a Julia thread, which must acquire the GIL using +Use this to run non-Python code with the GIL unlocked, so allowing another thread to run +Python code. That other thread can be a Julia thread, which must lock the GIL using [`lock`](@ref). -See [`@release`](@ref) for the macro form. +See [`@unlock`](@ref) for the macro form. """ -function release(f) +function unlock(f) state = C.PyEval_SaveThread() try f() @@ -72,17 +72,17 @@ function release(f) end """ - @release expr + @unlock expr -Release the GIL, compute `expr`, re-acquire the GIL, then return the result of `expr`. +Unlock the GIL, compute `expr`, re-lock the GIL, then return the result of `expr`. -Use this to run non-Python code with the GIL released, so allowing another thread to run -Python code. That other thread can be a Julia thread, which must acquire the GIL using +Use this to run non-Python code with the GIL unlocked, so allowing another thread to run +Python code. That other thread can be a Julia thread, which must lock the GIL using [`@lock`](@ref). -The macro equivalent of [`release`](@ref). +The macro equivalent of [`unlock`](@ref). """ -macro release(expr) +macro unlock(expr) quote state = C.PyEval_SaveThread() try diff --git a/src/JlWrap/any.jl b/src/JlWrap/any.jl index 573ca1a4..8ea73907 100644 --- a/src/JlWrap/any.jl +++ b/src/JlWrap/any.jl @@ -55,12 +55,12 @@ function pyjlany_call_nogil(self, args_::Py, kwargs_::Py) if pylen(kwargs_) > 0 args = pyconvert(Vector{Any}, args_) kwargs = pyconvert(Dict{Symbol,Any}, kwargs_) - ans = Py(GIL.@release self(args...; kwargs...)) + ans = Py(GIL.@unlock self(args...; kwargs...)) elseif pylen(args_) > 0 args = pyconvert(Vector{Any}, args_) - ans = Py(GIL.@release self(args...)) + ans = Py(GIL.@unlock self(args...)) else - ans = Py(GIL.@release self()) + ans = Py(GIL.@unlock self()) end pydel!(args_) pydel!(kwargs_) diff --git a/src/JlWrap/raw.jl b/src/JlWrap/raw.jl index 21e0fa7e..501f2aef 100644 --- a/src/JlWrap/raw.jl +++ b/src/JlWrap/raw.jl @@ -44,12 +44,12 @@ function pyjlraw_call_nogil(self, args_::Py, kwargs_::Py) if pylen(kwargs_) > 0 args = pyconvert(Vector{Any}, args_) kwargs = pyconvert(Dict{Symbol,Any}, kwargs_) - ans = pyjlraw(GIL.@release self(args...; kwargs...)) + ans = pyjlraw(GIL.@unlock self(args...; kwargs...)) elseif pylen(args_) > 0 args = pyconvert(Vector{Any}, args_) - ans = pyjlraw(GIL.@release self(args...)) + ans = pyjlraw(GIL.@unlock self(args...)) else - ans = pyjlraw(GIL.@release self()) + ans = pyjlraw(GIL.@unlock self()) end pydel!(args_) pydel!(kwargs_) diff --git a/test/GC.jl b/test/GC.jl index cc8d6ff0..84aa8477 100644 --- a/test/GC.jl +++ b/test/GC.jl @@ -1,7 +1,7 @@ @testitem "GC.gc()" begin let pyobjs = map(pylist, 1:100) - PythonCall.GIL.@release Threads.@threads for obj in pyobjs + PythonCall.GIL.@unlock Threads.@threads for obj in pyobjs finalize(obj) end end @@ -13,7 +13,7 @@ end @testitem "GC.GCHook" begin let pyobjs = map(pylist, 1:100) - PythonCall.GIL.@release Threads.@threads for obj in pyobjs + PythonCall.GIL.@unlock Threads.@threads for obj in pyobjs finalize(obj) end end diff --git a/test/GIL.jl b/test/GIL.jl index a0249282..ca1f6405 100644 --- a/test/GIL.jl +++ b/test/GIL.jl @@ -1,8 +1,8 @@ -@testitem "release and lock" begin - # This calls Python's time.sleep(1) twice concurrently. Since sleep() releases the +@testitem "unlock and lock" begin + # This calls Python's time.sleep(1) twice concurrently. Since sleep() unlocks the # GIL, these can happen in parallel if Julia has at least 2 threads. function threaded_sleep() - PythonCall.GIL.release() do + PythonCall.GIL.unlock() do Threads.@threads for i = 1:2 PythonCall.GIL.lock() do pyimport("time").sleep(1) @@ -20,11 +20,11 @@ end end -@testitem "@release and @lock" begin - # This calls Python's time.sleep(1) twice concurrently. Since sleep() releases the +@testitem "@unlock and @lock" begin + # This calls Python's time.sleep(1) twice concurrently. Since sleep() unlocks the # GIL, these can happen in parallel if Julia has at least 2 threads. function threaded_sleep() - PythonCall.GIL.@release Threads.@threads for i = 1:2 + PythonCall.GIL.@unlock Threads.@threads for i = 1:2 PythonCall.GIL.@lock pyimport("time").sleep(1) end end