test: Unit Tests for triton {metrics, config, status}
#66
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rmccorm4
reviewed
May 30, 2024
rmccorm4
reviewed
May 30, 2024
rmccorm4
reviewed
May 30, 2024
rmccorm4
reviewed
May 30, 2024
rmccorm4
reviewed
May 30, 2024
rmccorm4
reviewed
May 30, 2024
rmccorm4
reviewed
May 30, 2024
| def test_triton_status(self, model, setup_and_teardown): | ||
| pid = utils.run_server(repo=MODEL_REPO) # Import the Model | ||
| setup_and_teardown.pid = pid | ||
| utils.wait_for_server_ready() |
There was a problem hiding this comment.
For this PR, this is good - don't change it here.
However, some future food for thought - these tests that do (1) server start + (2) utility command should probably be in test_e2e.py. For test_cli.py it would probably be nice to have some super simple mock server implementation that can be used to test each of these commands standalone (more unit test style).
rmccorm4
approved these changes
May 30, 2024
rmccorm4
changed the title
triton {metrics, config, status}triton {metrics, config, status}
|
Nice job merging your first PR already! 🚀 |
KrishnanPrash
added a commit
that referenced
this pull request
Jul 12, 2024
* Psuedocode for the testing metrics * Blank Check * Fixing dependency issue * Skeleton Code * Dummy Prompt for inference * MVP for testing * Updting dependency * MVP for testing * MVP for testing * Standardizing JSON outputs * MVP for testing * Removing unnecessary model parameter * applying pre-commit linting changes and removing unused imports * Making inference check stricter * Removing test_model from and for consistency * Final commit * Fix dependency issues
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR is raised to address Jira Ticket [DLIS-6264].
Unit tests have been added primarily within
triton_cli/tests/test_cli.pyfortriton metrics,triton config, andtriton status.Brief overview of the tests:
For metrics:
test_modelsrepository,mock_llmis loaded.triton metricsoutput checked to see ifnv_inference_request_successreflects successful inference.For config:
test_modelsrepository,add_subandmock_llmare loaded.triton config -m model_nameis called.namefield of the returnedjsonis cross-referenced with the originalmodel_nameFor status:
test_modelsrepository,add_subandmock_llmare loaded.triton statusis called and verifies from output that the models areliveandready.Secondary Changes that were made while implementing the tests:
grpcio>=1.64.0topyproject.toml.triton_cli: Triton commands were previously returning keys in single quotes and boolean values as{True, False}instead of{true,false}. This leads to errors when the output is attempted to be parsed as a json.