[16.0][FIX] fastapi: Avoid zombie threads #486
Conversation
Each time a fastapi app is created, a new event loop thread is created by the ASGIMiddleware. Unfortunately, every time the cache is cleared, a new app is created with a new even loop thread. This leads to an increase in the number of threads created to manage the asyncio event loop, even though many of them are no longer in use. To avoid this problem, the thread in charge of the event loop is now created only once per thread / process and the result is stored in the thread's local storage. If a new instance of an app needs to be created following a cache reset, this ensures that the same event loop is reused. refs OCA#484
|
ping @sebalix |
fastapi/models/fastapi_endpoint.py
Outdated
| # Using a thread-local storage allows to have a dedicated event loop per thread | ||
| # and avoid the need to create a new event loop for each request. It's also | ||
| # compatible with the multi-worker mode of Odoo. | ||
| _event_loop_storage = threading.local() |
There was a problem hiding this comment.
Indeed, having one loop per thread is mandatory, missed this in #485 👍
EDIT: changing my mind, not working as expected, see below ⬇️
There was a problem hiding this comment.
I still have doubt here: this will attach the loop to the current thread, which is OK in workers mode as the current thread is always the main thread of the worker, but in multi-threads, that would mean we will have one loop per spawned thread to handle the user's request, which is not the main thread. Am I wrong?
I try to test all of this locally.
There was a problem hiding this comment.
Oh, is Odoo spawning a new thread for each request in multithread mode?
There was a problem hiding this comment.
What I did to reproduce this issue:
-
Here is Odoo running in multi-threads (2 exactly, forced with
ODOO_MAX_HTTP_THREADS+ cron threads disabled, and without queue_job neither), so we have the main thread+2 = 3:
-
Request done on a FastAPI endpoint (e.g.
curl http://localhost:8069/{root_path}/sales), one daemon thread spawned to handle the loop as expected:
-
Beside in a Odoo shell, I reset the cache and relaunch a request on FastAPI endpoint:
>>> env.registry.clear_caches() >>> env.registry.signal_changes()
curl http://localhost:8069/{root_path}/sales -
Repeating step 3. several times: I get a new thread each time, so we are not re-using the loop on requests, while it was working as expected with [16.0][FIX] fastapi: prevent the creation of multiple event loops/threads #485 because having the loop created at the module level was done only once in the main thread, not for each subsequent thread created by Odoo to handle user's requests.
There was a problem hiding this comment.
@sebalix does that mean sharing the same event loop across threads? Is that safe?
There was a problem hiding this comment.
Oh, is Odoo spawning a new thread for each request in multithread mode?
Yes: https://github.com/odoo/odoo/blob/18.0/odoo/service/server.py#L205-L217
@sebalix does that mean sharing the same event loop across threads? Is that safe?
I don't know... 🤷♂️ maybe not something supported by default (reading https://www.neilbotelho.com/blog/multithreaded-async.html ).
And what if we attach the event loop to the current thread? Or one not with daemon=True? So we get one event loop created for each user's request, does it imply a lot of overhead? (and we maybe lose the benefit of FastAPI by doing so)
|
@sbidoul additional expert advice would be welcome 😏 |
There was a problem hiding this comment.
Not approving based on my test here: https://github.com/OCA/rest-framework/pull/486/files#r1908260365
|
@sbidoul @sebalix Code will folllow |
|
@sebalix @sbidoul I've implemented a new approach based on queue.Queue to manage a dynamic pool of event loops and the fastapi application cache. I'm probably still missing a bit of code for shutting down the threads used for the event pool when the server stops. I haven't yet figured out how to implement this, even though the method is available on the pool to stop all the threads. |
lmignon
changed the title
This commit adds event loop lifecycle management to the FastAPI dispatcher. Before this commit, an event loop and the thread to run it were created each time a FastAPI app was created. The drawback of this approach is that when the app was destroyed (for example, when the cache of app was cleared), the event loop and the thread were not properly stopped, which could lead to memory leaks and zombie threads. This commit fixes this issue by creating a pool of event loops and threads that are shared among all FastAPI apps. On each call to a FastAPI app, a event loop is requested from the pool and is returned to the pool when the app is destroyed. At request time of an event loop, the pool try to reuse an existing event loop and if no event loop is available, a new event loop is created. The cache of the FastAPI app is also refactored to use it's own mechanism. It's now based on a dictionary of queues by root path by database, where each queue is a pool of FastAPI app. This allows a better management of the invalidation of the cache. It's now possible to invalidate the cache of FastAPI app by root path without affecting the cache of others root paths.
lmignon
force-pushed
the
16.0-fastapi-event-loop-lifecycle-lmi
branch
from
9ffda1a to
8c090cd
Compare
On server shutdown, ensure that created the event loops are closed properly.
lmignon
force-pushed
the
16.0-fastapi-event-loop-lifecycle-lmi
branch
from
f85ecbe to
289868d
Compare
defaultdict in python is not thread safe. Since this data structure is used to store the cache of FastAPI apps, we must ensure that the access to this cache is thread safe. This is done by using a lock to protect the access to the cache.
lmignon
force-pushed
the
16.0-fastapi-event-loop-lifecycle-lmi
branch
from
289868d to
1bf9751
Compare
Each time a fastapi app is created, a new event loop thread is created by the ASGIMiddleware. Unfortunately, every time the cache is cleared, a new app is created with a new even loop thread. This leads to an increase in the number of threads created to manage the asyncio event loop, even though many of them are no longer in use. To avoid this problem, the thread in charge of the event loop is now created only once per thread / process and the result is stored in the thread's local storage. If a new instance of an app needs to be created following a cache reset, this ensures that the same event loop is reused.
refs #484