Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
Clarification on how to run models requiring authentification
  • Loading branch information
Erechtheus authored Nov 14, 2024
1 parent b8fd229 commit d20dfd1
Showing 1 changed file with 7 additions and 2 deletions.
9 changes: 7 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,18 @@
# perseus-textgen
A repository for scripts to run awesomely large language models with text generation inference APIs and (chat) UIs

This repository contains scripts for running large language models using text generation inference APIs and (chat) UIs.

## Authentication
Certain models (e.g., Google-gemma, LLAMA3, etc.) require you to accept the license on the Hugging Face website. To download these models to your machine, you need an authentication token from [Hugging Face](https://huggingface.co/settings/tokens). To register your token on your machine, run the following command:

```sh
huggingface-cli login --token <yourToken>
```
For proper functionality, ensure the script has access to your API key, typically located at `$HOME/.cache/huggingface`. Use the following snippet to mount this directory in your srun command:

```sh
$HOME/.cache/huggingface:/root/.cache/huggingface
```

## Update the enroot image
Newer models sometimes require a never enroot image. To generate a new image adapt the following code-snippted.

Expand Down

0 comments on commit d20dfd1

Please sign in to comment.