py/scheduler: Only run callbacks on the main thread if GIL is disabled.

Otherwise it's very difficult to reason about thread safety in a
scheduler callback, as it can run at any time on any thread - including
racing against any bytecode operation on any thread.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <[email protected]>

docs/library/micropython.rst

@@ -136,6 +136,14 @@ Functions
    the heap may be locked) and scheduling a function to call later will lift
    those restrictions.
 
+   On multi-threaded ports, the scheduled function's behaviour depends on
+   whether the Global Interpreter Lock (GIL) is enabled for the specific port:
+
+   - If GIL is enabled, the function can preempt any thread and run in its
+     context.
+   - If GIL is disabled, the function will only preempt the main thread and run
+     in its context.
+
    Note: If `schedule()` is called from a preempting IRQ, when memory
    allocation is not allowed and the callback to be passed to `schedule()` is
    a bound method, passing this directly will fail. This is because creating a

py/scheduler.c

@@ -236,7 +236,13 @@ void mp_handle_pending(bool raise_exc) {
 
     // Handle any pending callbacks.
     #if MICROPY_ENABLE_SCHEDULER
-    if (MP_STATE_VM(sched_state) == MP_SCHED_PENDING) {
+    bool run_scheduler = (MP_STATE_VM(sched_state) == MP_SCHED_PENDING);
+    #if MICROPY_PY_THREAD && !MICROPY_PY_THREAD_GIL
+    // Avoid races by running the scheduler on the main thread, only.
+    // (Not needed if GIL enabled, as GIL ensures thread safety here.)
+    run_scheduler = run_scheduler && mp_thread_is_main_thread();
+    #endif
+    if (run_scheduler) {
         mp_sched_run_pending();
     }
     #endif