-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix control casing #18
Conversation
We should add the same fixes for local inference to the validator's |
@CalebCourier from the tests that I ran, it seems to detect
prints |
@aaravnavani that's only the initial filter. It also needs to be handled here and potentially in filtering methods. As a side note, the fact that it's listing competitors as an |
I'm not seeing that behaviour locally. If I try to run that exact code snippet this is what I get: ValidationOutcome(
call_id='14844740304',
raw_llm_output='apple just released a new iPhone.',
validated_output='apple just released a new iPhone.',
reask=None,
validation_passed=True,
error=None
) |
validator/main.py
Outdated
@@ -159,7 +159,7 @@ def _inference_local(self, model_input: Any) -> List[List[str]]: | |||
doc = self.nlp(t) | |||
located_entities = [] | |||
for ent in doc.ents: | |||
if ent.text in competitors: | |||
if ent.text.lower() in [comp.lower() for comp in competitors]: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Small nit: extract the competitors list comprehension to a variable
This reverts commit cf5226f.
This PR fixes this issue.
For example, if the competitor that is passed in is "Apple" and we are validating the text
Fun fact about apple is that apple is both a fruit and company. apples come in many different colors and flavors. What if apple is a tech company that makes phones, computers, and tablets? apple is headquartered in Cupertino, California
, it properly identifies lowercaseapple
as a competitor.Curl command to test with fastapi setup running on
localhost
:gives output:
{"modelname":"en_core_web_trf","modelversion":"1","outputs":[{"name":"result0","shape":[1],"data":[["apple","apple","apple"]],"datatype":"BYTES"}]}