forked from reanahub/reana-workflow-controller
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat(opensearch): capture logs from Dask cluster pods (reanahub#616)
This commit collects logs from Dask scheduler and workers and propagates them to all REANA jobs that are using the same Dask cluster. This is not ideal, since Dask logs can become thusly duplicated for different workflow steps of the workflow, which could be confusing for the user. However, when a user uses Dask to parallelise the workflow jobs, usually the workflow steps are defined only within Dask, so this situation does not occur. Hence we can afford doing this in usual real-life conditions. Separating Dask scheduler and worker logs from regular Kubernetes job logs would require a larger architectural change and is therefore deferred to a future commit. Closes reanahub#610
- Loading branch information
1 parent
afd1400
commit 51fad95
Showing
2 changed files
with
116 additions
and
14 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters