-
Notifications
You must be signed in to change notification settings - Fork 105
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve input mismatch error for inference requests (DLIS-6165) #330
Conversation
The message can be improved for better clarity and readability to something similar to the one below. [request ID: <id_unknown>] Error: Expected 5 inputs for model 'llama2-7b', but received only 1 input(s). |
Fixed this made the message more verbose. |
Please update the copyright year on both files, because this change is more than fixing a typo, i.e. on infer_request.h, the copyright should start with
same for infer_request.cc |
Fixed for all files |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great!
Created 2 new strings to improve the verbosity of the error response.
Scenario: When a inference request has less input params than the model expects error response is insufficient.
Fix: Added 2 new strings to the error response
Old Behavior:
[request id: <id_unknown>] expected 5 inputs but got 1 inputs for model 'llama2-7b'
New Behavior:
[request id: <id_unknown>] expected 5 inputs but got 1 inputs for model 'llama2-7b'. Got inputs ['prompt'], but missing ['max_output_token', 'top_k', 'top_p', 'temperature']"
We have added text to existing response so that existing tests do not break.
Working on the test case for this response will add later.