Add tests/test_classifiers.py PyTests
#421
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Sarah Yurick <[email protected]>
Signed-off-by: Sarah Yurick <[email protected]>
Signed-off-by: Sarah Yurick <[email protected]>
Signed-off-by: Sarah Yurick <[email protected]>
Signed-off-by: Sarah Yurick <[email protected]>
sarahyurick
added
gpuci
Signed-off-by: Sarah Yurick <[email protected]>
sarahyurick
added
gpuci
Signed-off-by: Sarah Yurick <[email protected]>
sarahyurick
added
gpuci
Signed-off-by: Sarah Yurick <[email protected]>
sarahyurick
added
gpuci
Signed-off-by: Sarah Yurick <[email protected]>
sarahyurick
added
gpuci
Signed-off-by: Sarah Yurick <[email protected]>
Signed-off-by: Sarah Yurick <[email protected]>
sarahyurick
commented
Dec 13, 2024
tests/test_classifiers.py
Outdated
Comment on lines
102
to
104
| hugging_face_api_token = os.getenv("HUGGING_FACE_API_TOKEN") | ||
| if not hugging_face_api_token: | ||
| pytest.fail("HUGGING_FACE_API_TOKEN environment variable not set") |
There was a problem hiding this comment.
We grab the Hugging Face token here instead of using a @pytest.fixture. The reason for this is that if we used a PyTest fixture and the test failed for whatever reason, the Hugging Face token would be exposed in the logs.
There was a problem hiding this comment.
If there is not a short term fix here, we can probably skip this test using skipif .
@pytest.mark.skipif(
not os.getenv("HUGGING_FACE_API_TOKEN"),
reason="HUGGING_FACE_API_TOKEN environment variable is not set"
)
VibhuJawa
approved these changes
Dec 16, 2024
There was a problem hiding this comment.
This looks good to me. Thanks a ton for adding these tests @sarahyurick .
I think we should be good to merge this once we resolve how to pass in HF Secret
sarahyurick
commented
Dec 16, 2024
Signed-off-by: Sarah Yurick <[email protected]>
Signed-off-by: Sarah Yurick <[email protected]>
sarahyurick
added
gpuci
Signed-off-by: Sarah Yurick <[email protected]>
sarahyurick
added
gpuci
sarahyurick
commented
Dec 20, 2024
Comment on lines
+238
to
+253
| expected_pred = cudf.DataFrame( | ||
| { | ||
| "constraint_ct": [0.5586], | ||
| "contextual_knowledge": [0.0559], | ||
| "creativity_scope": [0.0825], | ||
| "domain_knowledge": [0.9803], | ||
| "no_label_reason": [0.0], | ||
| "number_of_few_shots": [0], | ||
| "prompt_complexity_score": [0.2783], | ||
| "reasoning": [0.0632], | ||
| "task_type_1": ["Code Generation"], | ||
| "task_type_2": ["Text Generation"], | ||
| "task_type_prob": [0.767], | ||
| "text": text, | ||
| } | ||
| ) |
There was a problem hiding this comment.
This passes on my remote machine (both my conda setup and my Docker setup) but is failing on our gpuCI. I think perhaps it is related to floating point errors on different machines? I will probably just add some rounding estimations so that it passes both locally for me and here on GitHub.
Signed-off-by: Sarah Yurick <[email protected]>
sarahyurick
added
gpuci
Signed-off-by: Sarah Yurick <[email protected]>
sarahyurick
added
gpuci
Signed-off-by: Sarah Yurick <[email protected]>
sarahyurick
added
gpuci
Signed-off-by: Sarah Yurick <[email protected]>
sarahyurick
added
gpuci
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #405.
Closes #419.