Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace Cached with Yolox-graphic-elements and drop Deplot #365

Merged
merged 41 commits into from
Feb 13, 2025

Conversation

edknv
Copy link
Collaborator

@edknv edknv commented Jan 21, 2025

Description

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@edknv edknv marked this pull request as ready for review February 11, 2025 22:49
Copy link

copy-pr-bot bot commented Feb 12, 2025

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's also go ahead and make the switch to the llama embedder. This is what I was testing on the single GPU deployment.

EMBEDDING_IMAGE=nvcr.io/nim/nvidia/llama-3.2-nv-embedqa-1b-v2
EMBEDDING_TAG=latest
EMBEDDING_NIM_MODEL_NAME=nvidia/llama-3.2-nv-embedqa-1b-v2

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Switched to llama in 41f9e7b.

Copy link
Collaborator

@randerzander randerzander left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please also add to top of README:

**NOTICE**:
Cached and Deplot are deprecated, docker-compose now points to a beta version of the yolox-graphic-elements container instead. That model and container is slated for full release in March.

With this change, you should now be able to run on a single 80GB A100 or H100 GPU.

and pre-requisites, move number of A100s and H100s to 1x

@edknv
Copy link
Collaborator Author

edknv commented Feb 13, 2025

Noitce added at the top of REAMDE and minimum requirements adjusted to 1 GPU.
fa6e750
fd65f2d
20c290f

@edknv edknv merged commit 32a9021 into NVIDIA:main Feb 13, 2025
2 of 3 checks passed
@edknv edknv deleted the edwardk/replace-cached-with-yolox-1 branch February 13, 2025 17:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants