(WIP) Dynamic inference provider mapping #1173
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
4 tasks
julien-c
reviewed
Feb 5, 2025
|
|
||
| type FalAiId = string; | ||
|
|
||
| export const FAL_AI_SUPPORTED_MODEL_IDS: ProviderMapping<FalAiId> = { |
There was a problem hiding this comment.
same comment as in https://github.com/huggingface/huggingface_hub/pull/2836/files#r1942755169, i would for now keep the mappings (but make them empty)
and still support them in code
That way we have slightly less breaking changes too, potentially
There was a problem hiding this comment.
@julien-c I reverted to keep both the hard-coded logic and the new dynamic mapping. Mapping from the Hub takes precedence over the hardcoded one.
|
Depends on https://github.com/huggingface-internal/moon-landing/pull/12453 (internal) |
SBrandeis
approved these changes
Feb 5, 2025
julien-c
reviewed
Feb 5, 2025
julien-c
reviewed
Feb 5, 2025
julien-c
reviewed
Feb 5, 2025
| `Model ${params.model} is in staging mode for provider ${params.provider}. Meant for test purposes only.` | ||
| ); | ||
| } | ||
| // TODO: how is it handled server-side if model has multiple tasks (e.g. `text-generation` + `conversational`)? |
There was a problem hiding this comment.
Suggested change
| // TODO: how is it handled server-side if model has multiple tasks (e.g. `text-generation` + `conversational`)? |
i think this is ok @Wauplin
julien-c
approved these changes
Feb 5, 2025
|
BG @SBrandeis 
|
SBrandeis
requested review from
gary149,
pcuenca,
ngxson and
hanouticelina
as code owners
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
"it should work" 😄
Still a draft while https://github.com/huggingface-internal/moon-landing/pull/12398 (internal) is been merged/deployed.
Goal is to use the dynamic mapping, and default back to hardcoded model ids if necessary (for backward compatibility).
I haven't tested anything for now and I left some todos to address:
status: "live" | "staging"? => raise a warningmodelInfocall (only do it once at runtime)text-generationandconversational"hf-inference"if no taskHints? (for now, I kept as before)EDIT: made an update to preserve previous behavior with hardcoded mapping. Dynamic mapping from the hub
takes precedence.