-
Notifications
You must be signed in to change notification settings - Fork 80
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[Cookbook] Add Anyscale Endpoint cookbook
Clone the Getting Started cookbook and adapt it for Anyscale. Next up we will also add a cookbook for function calling using Anyscale.
- Loading branch information
Showing
5 changed files
with
680 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
# Anyscale Endpoints with AIConfig | ||
|
||
[](https://colab.research.google.com/drive/1JgGjJ2YglyaT5GHQNswkPOyB5oHGbOcv?usp=sharing) | ||
|
||
[Anyscale Endpoints](https://www.anyscale.com/endpoints) support optimized inference for many open source models, including the LLaMA2 family of models (7B, 13B, 70B, CodeLLaMA, LLaMA Guard) and Mistral (7B, Mixtral 8x7B). | ||
|
||
This cookbook shows how to use any [Anyscale Endpoints](https://www.anyscale.com/endpoints) model with AIConfig using the same simple API. | ||
|
||
We cover: | ||
|
||
- Inference using Anyscale Endpoints | ||
- Prompt chains with multiple models | ||
|
||
Read more about [AIConfig for prompt and model management](https://github.com/lastmile-ai/aiconfig). | ||
|
||
## Models supported with Anyscale Endpoints | ||
|
||
For the complete list, please see https://app.endpoints.anyscale.com/docs | ||
|
||
- LLaMA-7B: meta-llama/Llama-2-7b-chat-hf | ||
- LLaMA-13B: meta-llama/Llama-2-13b-chat-hf | ||
- LLaMA-70B: meta-llama/Llama-2-70b-chat-hf | ||
- LLaMA Guard: Meta-Llama/Llama-Guard-7b | ||
- CodeLLaMA: codellama/CodeLlama-34b-Instruct-hf | ||
- Mistral-7B (OpenOrca): Open-Orca/Mistral-7B-OpenOrca | ||
- Mistral-7B: mistralai/Mistral-7B-Instruct-v0.1 | ||
- Mixtral-8x7B: mistralai/Mixtral-8x7B-Instruct-v0.1 | ||
- Zephyr: HuggingFaceH4/zephyr-7b-beta | ||
- GTE: thenlper/gte-large |
Oops, something went wrong.