[InferenceClient] Better handling of task parameters #2812
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
| @@ -2407,7 +2405,7 @@ def text_to_image( | |||
| scheduler: Optional[str] = None, | |||
| target_size: Optional[TextToImageTargetSize] = None, | |||
| seed: Optional[int] = None, | |||
| **kwargs, | |||
There was a problem hiding this comment.
This is a breaking change but hopefully totally fine
There was a problem hiding this comment.
i'll mention this in the next release notes! but it should be fine, if users were previously using text_to_image with the HF Inference API, this shouldn't be an issue since all API parameters were exposed as explicit method arguments
There was a problem hiding this comment.
this shouldn't be an issue since all API parameters were exposed as explicit method arguments
yes exactly
| @@ -2443,6 +2441,9 @@ def text_to_image( | |||
| The size in pixel of the output image | |||
| seed (`int`, *optional*): | |||
| Seed for the random number generator. | |||
| extra_parameters (`Dict[str, Any]`, *optional*): | |||
There was a problem hiding this comment.
Do we have a good example of how to use extra_parameters for a specific model on a specific provider? Would be good to add a least one example (either in text-to-image or text-to-video depending on what's best)
There was a problem hiding this comment.
yep, added in 305c720
| @pytest.mark.parametrize( | ||
| "helper,inputs,parameters,expected_data,expected_json", | ||
| [ |
| @@ -134,7 +134,7 @@ def __init__(self): | |||
|
|
|||
| def _prepare_payload(self, inputs: Any, parameters: Dict[str, Any]) -> Dict[str, Any]: | |||
| parameters = {k: v for k, v in parameters.items() if v is not None} | |||
| if "image_size" not in parameters and "width" in parameters and "height" in parameters: | |||
| if "width" in parameters and "height" in parameters: | |||
There was a problem hiding this comment.
what if only one if passed btw?
There was a problem hiding this comment.
we should be able to send only one if specified, the other one would be set to the default value. I'll fix that
There was a problem hiding this comment.
no actually for fal-ai, you either send both, or neither.
There was a problem hiding this comment.
there is no default values for each one of them, according to their documentation
|
should be good now, let's merge! |
There was a problem hiding this comment.
(nit) i think OpenAI calls this param extra_body (https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html#extra-parameters or from the Python openai sdk), but i think our name is probably fine too 🤷
|
Then I'm down to go with |
|
up to you (i'm not sure it's exactly the same use case, but my brain is fried) |
|
It's actually the same use case, you can send an extra parameter using openai client, but i'm not sure if it's |
|
extra_query goes into the URL query params no? (just guessing) |
|
yep, just wanted to double check in their docstrings, since it's not really documented. |
This PR:
text-to-imageparameters (particularly for Together AI).extra_parametersargument totext_to_image,text_to_speechandtext_to_videoto be able to pass a provider's unique parameters for these tasks.