Fix issue where hugging face response was not correct type #769
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rossdanlm
force-pushed
the
pr769
branch
2 times, most recently
from
e356f92 to
5d07265
Compare
rossdanlm
requested review from
saqadri,
rholinshead,
suyoglastmileai,
Ankush-lastmile and
jonathanlastmileai
as code owners
rossdanlm
pushed a commit
that referenced
this pull request
Jan 5, 2024
Sometimes key is not defined or set, we need to do this to unblock #769 The functionality is still unchanged because the optional param `required` defaults to true ## Test Plan Pass all automated tests (which it didn't before)
rossdanlm
pushed a commit
that referenced
this pull request
Jan 5, 2024
Sometimes key is not defined or set, we need to do this to unblock #769 The functionality is still unchanged because the optional param `required` defaults to true ## Test Plan Pass all automated tests (which it didn't before)
rossdanlm
pushed a commit
that referenced
this pull request
Jan 5, 2024
Sometimes key is not defined or set, we need to do this to unblock #769 The functionality is still unchanged because the optional param `required` defaults to true ## Test Plan Pass all automated tests (which it didn't before)
rholinshead
approved these changes
Jan 5, 2024
saqadri
approved these changes
Jan 5, 2024
cookbooks/HuggingFace/hf.py
Outdated
| token = maybe_get_api_key_from_environment("HUGGING_FACE_API_TOKEN") | ||
| # You are allowed to use Hugging Face for a bit before you get | ||
| # rate limited, in which case you will receive a clear error | ||
| token = maybe_get_api_key_from_environment("HUGGING_FACE_API_TOKEN", required=False) |
There was a problem hiding this comment.
I left a comment, but it's because Hugging Face allows you to use their API for a bit before you get rate-limited. When you get rate limited, we get a very clear error from Hugging Face saying to login and use an API key
| @@ -120,7 +120,7 @@ class HuggingFaceTextGenerationParser(ParameterizedModelParser): | |||
| A model parser for HuggingFace text generation models. | |||
| """ | |||
|
|
|||
| def __init__(self, model_id: str = None, use_api_token=False): | |||
| def __init__(self, model_id: str = None, use_api_token=True): | |||
There was a problem hiding this comment.
Why was this false before and why is it set to True now?
There was a problem hiding this comment.
It's because after using it for a bit, you end up getting rate-limited, so this needs to be true so that you can by default fetch your API key
rossdanlm
pushed a commit
that referenced
this pull request
Jan 5, 2024
Sometimes key is not defined or set, we need to do this to unblock #769 The functionality is still unchanged because the optional param `required` defaults to true ## Test Plan Pass all automated tests (which it didn't before)
rossdanlm
force-pushed
the
pr769
branch
2 times, most recently
from
364c209 to
5055bec
Compare
rossdanlm
added a commit
that referenced
this pull request
Jan 5, 2024
Make `get_api_key_from_environment` return nullable Sometimes key is not defined or set, we need to do this to unblock #769 The functionality is still unchanged because the optional param `required` defaults to true ## Test Plan Pass all automated tests (which it didn't before) --- Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/lastmile-ai/aiconfig/pull/772). * #769 * __->__ #772
Turned out to be a typing issue that python did not actually help with. I also had to adjust the way we load in hugging face key because I got rate limited. We NEED to have more clear way to set API keys in `aiconfig/.env` file though, or else people won't know how to use it! cc @tanya-rai let's add an FAQ/README similar to what we have in the cookbooks: https://github.com/lastmile-ai/aiconfig/blob/3dcf70bcddda7f7041c3bfb6e5d4b6c9a7abd4ab/cookbooks/Getting-Started/getting_started.ipynb?short_path=dcb1340#L23-L25 We need to do this for keys - HUGGING_FACE_API_TOKEN - ANYSCALE_ENDPOINT_API_KEY - OPENAI_API_KEY - GOOGLE_API_KEY ## Testing Connected to local editor (you'll have to modify the `max_tokens` setting) and runs now. We should also have automated testing for this hugging face model parser. Created task for it in #768 <img width="1512" alt="Screenshot 2024-01-05 at 00 47 51" src="https://github.com/lastmile-ai/aiconfig/assets/151060367/5ecae48a-c0cd-43cf-a7ed-ccc74929c6b3">
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fix issue where hugging face response was not correct type
Turned out to be a typing issue that python did not actually help with.
I also had to adjust the way we load in hugging face key because I got rate limited. We NEED to have more clear way to set API keys in
aiconfig/.envfile though, or else people won't know how to use it! cc @tanya-rai let's add an FAQ/README similar to what we have in the cookbooks:aiconfig/cookbooks/Getting-Started/getting_started.ipynb
Lines 23 to 25 in 3dcf70b
We need to do this for keys
Testing
Connected to local editor (you'll have to modify the
max_tokenssetting) and runs now. We should also have automated testing for this hugging face model parser. Created task for it in #768