-
Notifications
You must be signed in to change notification settings - Fork 219
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Replace Cached with Yolox-graphic-elements and drop Deplot #365
Replace Cached with Yolox-graphic-elements and drop Deplot #365
Conversation
…v/nv-ingest into edwardk/replace-cached-with-yolox-1
…v/nv-ingest into edwardk/replace-cached-with-yolox-1
…v/nv-ingest into edwardk/replace-cached-with-yolox-1
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's also go ahead and make the switch to the llama embedder. This is what I was testing on the single GPU deployment.
EMBEDDING_IMAGE=nvcr.io/nim/nvidia/llama-3.2-nv-embedqa-1b-v2
EMBEDDING_TAG=latest
EMBEDDING_NIM_MODEL_NAME=nvidia/llama-3.2-nv-embedqa-1b-v2
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Switched to llama in 41f9e7b.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please also add to top of README:
**NOTICE**:
Cached and Deplot are deprecated, docker-compose now points to a beta version of the yolox-graphic-elements container instead. That model and container is slated for full release in March.
With this change, you should now be able to run on a single 80GB A100 or H100 GPU.
and pre-requisites, move number of A100s and H100s to 1x
Description
Checklist