Add opentelemetry integration #467
Conversation
Add a draft opentelemtry integration consisting of a span propagator, a decorator for instrumenting jobs and a class for hooking into enqueue_job with span propagation. Additionally there are some metrics for the number of enqueued jobs. This is a reasonably bare-bones implementation that needs work in a few areas which I will can come back to, but I think there are a few weaknesses I can think of: * Having to explicitly instrument jobs using the decorator is a bit clunky and perhaps it should happen automatically? * `InstrumentedArqRedis` is a bit awkward to use and differs quite a lot from the `opentelemetry-contrib` modules which use Instrumentors. * I'm not certain about whether this should be an extra to the main `arq` package or a separate extension.
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #467 +/- ##
===========================================
- Coverage 96.27% 83.73% -12.54%
===========================================
Files 11 16 +5
Lines 1074 1242 +168
Branches 209 211 +2
===========================================
+ Hits 1034 1040 +6
- Misses 19 182 +163
+ Partials 21 20 -1
Continue to review full report in Codecov by Sentry.
|
|
Codecov ReportAttention: Patch coverage is ❗ Your organization needs to install the Codecov GitHub app to enable full functionality. @@ Coverage Diff @@
## main #467 +/- ##
===========================================
- Coverage 96.27% 83.73% -12.54%
===========================================
Files 11 16 +5
Lines 1074 1242 +168
Branches 209 211 +2
===========================================
+ Hits 1034 1040 +6
- Misses 19 182 +163
+ Partials 21 20 -1
Continue to review full report in Codecov by Sentry.
|
Add a draft opentelemetry integration consisting of a span propagator, a decorator for instrumenting jobs and a class for hooking into enqueue_job with span propagation. Additionally there are some metrics for the number of enqueued jobs.
This is a reasonably bare-bones implementation that needs work in a few areas which I will can come back to, but I think there are a few weaknesses I can think of:
InstrumentedArqRedisis a bit awkward to use and differs quite a lot from theopentelemetry-contribmodules which use Instrumentors.arqpackage or a separate extension.edit: Obviously this also needs me to write some tests and docs...
I should also mention that this is adapted from our internal code base at Quantinuum, where both @isobelhooper and myself have been working on this intermittently over the past few years.