-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Docker run support for HF_TOKEN authentication using env var pass in #23
Labels
enhancement
New feature or request
Comments
tstescoTT
added a commit
that referenced
this issue
Jan 14, 2025
… HF download change log: - add multiple model support using persistent_volume/model_envs/*.env - setup using Hugging Face huggingface-cli to download models: llama model install script support for llama CLI and huggingface hub #14 - add model setup for llama 3.x - address Initial vLLM setup fails due to missing HuggingFace permissions #37 - address Docker run support for HF_TOKEN authentication using env var pass in #23 - renamed vllm-tt-metal-llama3-70 to vllm-tt-metal-llama3 for all llama 3.x models - updated documentation for v0 drop - add Docker Ubuntu 22.04 option for vLLM llama 3.x
tstescoTT
added a commit
that referenced
this issue
Jan 15, 2025
… HF download (#67) * Llama 3.x model support, setup.sh script multiple model support using HF download change log: - add multiple model support using persistent_volume/model_envs/*.env - setup using Hugging Face huggingface-cli to download models: llama model install script support for llama CLI and huggingface hub #14 - add model setup for llama 3.x - address Initial vLLM setup fails due to missing HuggingFace permissions #37 - address Docker run support for HF_TOKEN authentication using env var pass in #23 - renamed vllm-tt-metal-llama3-70 to vllm-tt-metal-llama3 for all llama 3.x models - updated documentation for v0 drop - add Docker Ubuntu 22.04 option for vLLM llama 3.x * use vllm.llama3.src.shared.Dockerfile for shared build steps for ubuntu 22.04 and 20.04 Dockerfiles * use full url TT_METAL_DOCKERFILE_URL to allow for 1 Dockerfile for multiple base images
Addressed in #67 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
To keep convention with vLLM Docker containers described in https://github.com/vllm-project/vllm/blob/main/docs/source/serving/deploying_with_docker.rst, e.g.:
HUGGING_FACE_HUB_TOKEN
is being deprecated in favor ofHF_TOKEN
as defined in https://huggingface.co/docs/huggingface_hub/en/package_reference/environment_variables#deprecated-environment-variables, so it's recommended to useHF_TOKEN
here.This could be used to allow users to download models without having to enter credentials manually where possible.
The text was updated successfully, but these errors were encountered: