Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deployment of agent container is always attempted #195

Open
bgoelTT opened this issue Feb 24, 2025 · 4 comments
Open

Deployment of agent container is always attempted #195

bgoelTT opened this issue Feb 24, 2025 · 4 comments
Assignees
Labels
bug Something isn't working

Comments

@bgoelTT
Copy link
Contributor

bgoelTT commented Feb 24, 2025

Describe the bug
In app/api/docker_control/views.py we also attempt to deploy the agent container for every model. This does not make sense for non-LLM models, causing the API call to "fail" but the actual model deployment is successful.

To Reproduce
Steps to reproduce the behavior:

  1. Go to 'deploy YOLOv4 inference server'
  2. Deploy button will fail and turn red, but inference server was launched successfully

Expected behavior
We should launch the agent container depending on the model type. This is an exact scenario where implementing #167 will fix all of our problems. I think the solution from #167 should be when deciding whether to deploy the agent container.

@bgoelTT bgoelTT added the bug Something isn't working label Feb 24, 2025
@bgoelTT
Copy link
Contributor Author

bgoelTT commented Feb 24, 2025

@mvanniasingheTT which models are compatible for using with the agent container? I assume its all the LLMs?

@mvanniasingheTT
Copy link
Contributor

@mvanniasingheTT which models are compatible for using with the agent container? I assume its all the LLMs?

All the llama models except the small ones like 3B should work.

@mvanniasingheTT mvanniasingheTT self-assigned this Feb 24, 2025
@bgoelTT
Copy link
Contributor Author

bgoelTT commented Feb 24, 2025

I don't mean work in terms of accuracy. I mean do they raise errors, if any? I expect the small LLMs like 3.1-1B to still deploy properly but are kind of buggy.

@mvanniasingheTT
Copy link
Contributor

Yes, that's correct. All the LLMs, even the small ones, still deploy properly.

mvanniasingheTT added a commit that referenced this issue Feb 25, 2025
* add ModelType to address #167

* fix syntax issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants