Skip to content

Commit

Permalink
Restructure documentation, use phi-3-mini, include structured generat…
Browse files Browse the repository at this point in the history
…ion example, add feature matrix to models
  • Loading branch information
lapp0 committed Jul 22, 2024
1 parent a7e3381 commit 6bf9849
Show file tree
Hide file tree
Showing 30 changed files with 691 additions and 322 deletions.
157 changes: 157 additions & 0 deletions docs/assets/images/logits_processing_diagram.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
6 changes: 4 additions & 2 deletions docs/cookbook/deploy-using-modal.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ In this guide we will show you how you can use Modal to run programs written wit

## Build the image

First we need to define our container image. We download the Mistral-7B-v0.1 model from HuggingFace as part of the definition of the image so it only needs to be done once.
First we need to define our container image. We download the Mistral-7B-v0.1 model from HuggingFace as part of the definition of the image so it only needs to be done once (you need to provide an [access token](https://huggingface.co/settings/tokens))

```python
from modal import Image, App, gpu
Expand All @@ -21,6 +21,8 @@ outlines_image = Image.debian_slim(python_version="3.11").pip_install(
)

def import_model():
import os
os.environ["HF_TOKEN"] = "YOUR_HUGGINGFACE_TOKEN"
import outlines
outlines.models.transformers("mistralai/Mistral-7B-Instruct-v0.2")

Expand Down Expand Up @@ -73,7 +75,7 @@ schema = """{
To make the inference work on Modal we need to wrap the corresponding function in a `@app.function` decorator. We pass to this decorator the image and GPU on which we want this function to run (here an A100 with 80GB memory):

```python
@app.function(image=outlines_image, gpu=gpu.A100(memory=80))
@app.function(image=outlines_image, gpu=gpu.A100(size='80GB'))
def generate(
prompt: str = "Amiri, a 53 year old warrior woman with a sword and leather armor.",
):
Expand Down
2 changes: 2 additions & 0 deletions docs/cookbook/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,5 @@
- [Chain Of Density](chain_of_density.md): Summarize documents using chain of density prompting and JSON-structured generation.
- [Playing Chess](models_playing_chess.md): Make Mistral-7B play chess against itself using regex-structured generation.
- [SimToM](simtom.md): Improve LLMs' Theory of Mind capabilities with perspective-taking prompting and JSON-structured generation.
- [Q&A with Citations](qa-with-citations.md): Answer questions and provide citations using JSON-structured generation.
- [Knowledge Graph Generation](knowledge_graph_extraction.md): Generate a Knowledge Graph from unstructured text using JSON-structured generation.
Loading

0 comments on commit 6bf9849

Please sign in to comment.