Switch to sampling profiler for driver profiling #1399
Labels
enhancement
Improves the status quo
help wanted
We'd be happy about a community contribution
:Load Driver
Changes that affect the core of the load driver such as scheduling, the measurement approach etc.
Milestone
Profiling the load driver is important to make sure there's no accidental client bottleneck. Rally currently uses yappi for this: https://esrally.readthedocs.io/en/latest/command_line_reference.html#clr-enable-driver-profiling.
My understanding is that yappi is basically
cProfile
with asyncio support. It's a deterministic tracing profiler, which means that every single function call is recorded. In practice, just like cProfile, it's going to tell us functions are slow because the profiler is making them slow. This is whycProfile
is basically a bad practice in Python.We should use a sampling profiler instead, like vmprof or py-spy. vmprof appears to be useful for async/await code: python-trio/trio#943. It would be nice if profiling stayed as easy as it is now, but producing useful results is even more important!
The text was updated successfully, but these errors were encountered: