Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce not given and apply it to dataset's target #1612

Merged
merged 21 commits into from
Nov 29, 2023

Conversation

kevinmessiaen
Copy link
Member

Description

Introduce not given and apply it to dataset's target

Related Issue

Type of Change

  • 📚 Examples / docs / tutorials / dependencies update
  • 🔧 Bug fix (non-breaking change which fixes an issue)
  • 🥂 Improvement (non-breaking change which improves an existing feature)
  • 🚀 New feature (non-breaking change which adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to change)
  • 🔐 Security fix

Checklist

  • I've read the CODE_OF_CONDUCT.md document.
  • I've read the CONTRIBUTING.md guide.
  • I've updated the code style using make codestyle.
  • I've written tests for all new methods and classes that I created.
  • I've written the docstring in Google format for all the methods and classes that I used.

@kevinmessiaen kevinmessiaen marked this pull request as ready for review November 16, 2023 10:35
Copy link
Contributor

@rabah-khalek rabah-khalek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me @kevinmessiaen.

Could we also undo the changes done in https://github.com/Giskard-AI/giskard/pull/1531/files (the ones related to suppress the warning of target for LLM) and add the target=None in the BaseModel for when model_type == "text_generation" to suppress the warning?

@kevinmessiaen
Copy link
Member Author

kevinmessiaen commented Nov 20, 2023

Looks good to me @kevinmessiaen.

Could we also undo the changes done in https://github.com/Giskard-AI/giskard/pull/1531/files (the ones related to suppress the warning of target for LLM) and add the target=None in the BaseModel for when model_type == "text_generation" to suppress the warning?

Not sure to understand the logic of the validation though, why to we validate the dataset target when validating model?

For me this validation is completely unrelated to the model validation and should be done when validating dataset only.
When uploading a model we should not confuse the user with warning that the dataset is missing a target, since it's unrelated.

@rabah-khalek
Copy link
Contributor

that's exactly what I meant by my comment. We should remove it.
We had to do it to suppress the warning only for LLMs. We needed to do it inside validate_model just to know the model_type back then. But now people can put explicitly target=None to supress this warning

@kevinmessiaen
Copy link
Member Author

that's exactly what I meant by my comment. We should remove it. We had to do it to suppress the warning only for LLMs. We needed to do it inside validate_model just to know the model_type back then. But now people can put explicitly target=None to supress this warning

Thanks for clarification, so it's what I did then, you can review the PR when you have time

@kevinmessiaen kevinmessiaen self-assigned this Nov 27, 2023
@kevinmessiaen kevinmessiaen removed their assignment Nov 27, 2023
kevinmessiaen and others added 2 commits November 28, 2023 09:09
# Conflicts:
#	docs/reference/notebooks/LLM_QA_Google.ipynb
#	giskard/core/model_validation.py
Copy link
Contributor

@rabah-khalek rabah-khalek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @kevinmessiaen, looks good

Copy link

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

96.3% 96.3% Coverage
0.0% 0.0% Duplication

@kevinmessiaen kevinmessiaen merged commit 204c1b0 into main Nov 29, 2023
17 checks passed
@kevinmessiaen kevinmessiaen deleted the feature/gsk-2118-introduce-not_given-type-2 branch November 29, 2023 15:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants