add ResourceJournalConsumer and EventLogFormatter classes and use them to standardize flux resource eventlog
#6614
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
grondo
force-pushed
the
resource-journal-consumer
branch
2 times, most recently
from
c2f2388 to
cbb5beb
Compare
garlick
approved these changes
Feb 8, 2025
| subclasses will follow the interface documented here. | ||
|
|
||
| A journal consumer is created by passing the constructor an open | ||
| :obj:`~flux.Flux` handle, the topic string of the journal servicealong |
grondo
force-pushed
the
resource-journal-consumer
branch
from
cbb5beb to
10502d9
Compare
Problem: The EventLogEvent class is defined in flux.job.event, but the eventlog format is not specific to jobs. Move the EventLogEvent class up a level to eventlog.py. For backwards compatibility, retain the ability to import EventLogEvent from flux.job and flux.job.event.
Problem: The JournalConsumer class in flux.job is specific to the job manager journal, but there may be other services that implement a journal-like interface. Add an abstract base class for journal consumer interfaces in flux.abc.JournalConsumerBase. Include a JournalEventBase class that derives from EventLogEvent. Most of the journal consumer interface is implemented in this class, with the expectation that derived classes only need to implement the few abstract methods left open for implementations.
Problem: The flux.job.JournalConsumer class duplicates code from the JournalConsumerBase class. Make flux.job.JournalConsumer a subclass of the JournalConsumerBase abstract base class.
Problem: There is no interface for the resource module journal. Add flux.resource.journal.JournalConsumer as a subclass of the JournalConsumerBase abstract base class.
Problem: There are no test of the Python ResourceJournalConsumer class. Add a Python based test in t/python/t0032-resource-journal.py.
Problem: Python utilities do not have access to the eventlog formatter class used by all C utilities for standard eventlog presentation. Add a Python EventLogFormatter class that mimics the C version.
Problem: There are no tests of the Python EventLogFormatter class. Add a new test t/python/t0033-eventlog-formatter.py.
Problem: `flux resource eventlog` doesn't have the same options and functionality as other eventlog commands. Make the options and functionality of `flux resource eventlog` match the standard Flux utility eventlog interface using the Python EventLogFormatter class.
Problem: t2355-resource-journal.t assumes the output of `flux resource eventlog` is JSON, but this is no longer true. Pass `-f json` to the command to force JSON output.
Problem: Newly added options to `flux resource eventlog` are not tested in t2355-resource-journal.t. Expand testing to cover new options.
Problem: The `flux resource eventlog` implementation uses the resource.journal RPC directly, but there's now a Python class for that. Use ResourceJournalConsumer to simplify the `flux resource eventlog` implementation.
Problem: New options to `flux resource eventlog` are not documented in flux-resource(1). Update the docs.
Problem: The `eventlog` subcommand of flux-resource(1) does not have tab completions. Add them to etc/completions/flux.pre.
grondo
force-pushed
the
resource-journal-consumer
branch
from
10502d9 to
90ef568
Compare
|
Thanks! I've rebased and fixed that typo and will set MWP now. |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #6614 +/- ##
==========================================
+ Coverage 83.81% 83.86% +0.05%
==========================================
Files 530 534 +4
Lines 88237 88381 +144
==========================================
+ Hits 73955 74120 +165
+ Misses 14282 14261 -21
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The main purpose of this PR is to add the Python
ResourceJournalConsumerclass, which mimics the existing job managerJournalConsumerclass. In fact, most of the originalJournalConsumerclass implementation is moved to an abstractJournalConsumerBaseclass, then both the job managerJournalConsumerandResourceJournalConsumerclasses inherit from that.In addition, the
flux resource eventlogutility is modified to have similar options to the rest of theeventlogutilities in Flux (--format,--time-format,--color,--human)using a newEventLogFormatterclass and the newly addedResourceJournalConsumerclass.