python: add AccountingFormatter class, SQLite utility file
#520
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cmoussa1
force-pushed
the
add.formatter.class
branch
3 times, most recently
from
edd543b to
d3b5538
Compare
cmoussa1
changed the title
AccountingFormatter class, restructure list-banks to use new classAccountingFormatter class, SQLite utility file
cmoussa1
force-pushed
the
add.formatter.class
branch
from
d3b5538 to
7e44485
Compare
Problem: flux-accounting has lots of different tables that can be viewed with view-* and list-* commands, but each one has its own function that could probably be cleaned up with some common utility. Begin to clean up this code by creating a new AccountingFormatter class. Take a SQLite Cursor object (the object responsible for executing a SQL query) as part of initialization. Define some formatting methods for this class, such as as_table and as_json(), which will take the Cursor object and format its result in table format or JSON format, respectively, as well as some basic metadata methods, like get_column_names() or get_rows().
Problem: the flux-accounting Python bindings have lots of separate (and probably duplicate) helper code for validating and interacting with the SQLite database. It would be cleaner and easier to maintain if it was centralized in one file that could be referenced throughout the bindings. Add a new SQL utility file to the Python bindings that will have helper functions for interacting with the SQLite database. Add a function to this file that will validate a custom list of table column names against a valid list of table column names.
Problem: The default fields for the relevant tables in the flux-accounting DB are defined manually throughout the bindings, so it makes sense to define these globally in the accounting Python fluxacct.accounting package. Create a definition for the names of the columns in some of the tables in __init__.py so that they can be referenced throughout the Python bindings for flux-accounting.
Problem: flux-accounting does not have tests that use the common formatter in the Python bindings. Add some unit tests for construction of AccountingFormatter objects and ensuring the correct amount of data is returned from executing a query. Test that column names can be fetched from AccountingFormatter objects and are consistent with the ones defined in the flux-accounting DB. Test that the correct exceptions are raised when invalid column names are passed through validation.
cmoussa1
force-pushed
the
add.formatter.class
branch
from
7e44485 to
1432f8e
Compare
wihobbs
approved these changes
Oct 29, 2024
|
Great suggestion @wihobbs! I'll open an issue on getting a |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
flux-accounting has lots of different tables that can be viewed with
view-*andlist-*commands, but each one has its own function that could probably be cleaned up with some common utility. Better yet, if the formatting options for all of these commands was centralized, it might make for easier changes in the future and/or customization of output by users.This PR begins to centralize the formatting for flux-accounting by adding a new
AccountingFormatterclass to the Python bindings. The class takes a SQLiteCursorobject that has the results of running a query on the flux-accounting database and offers two methods of formatting its data: data in an adjustable table or data in JSON format. Since this class just takes aCursorobject, this means it can support customization of what fields are formatted and returned (because the query itself can just be customized to not fetch every field).I've also added a SQLite utility file to the Python bindings to also help centralize some of the basic SQL helpers I have throughout the bindings. I've started with just a simple validation function where, given a custom list of columns passed in, that list is then validated against what is actually defined in the flux-accounting database.
I've added some basic unit tests for the
AccountingFormatterclass and the utilization of thesql_utilhelper function.