Improve vLLM backend documentation #22
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tanmayv25
force-pushed
the
tanmayv-doc
branch
from
6b20675 to
9879a48
Compare
Co-authored-by: Olga Andreeva <[email protected]>
Co-authored-by: Olga Andreeva <[email protected]>
oandreeva-nv
previously approved these changes
Nov 22, 2023
dyastremsky
reviewed
Nov 22, 2023
README.md
Outdated
| registry. These are available starting in 23.10. | ||
| The tritonserver_vllm container has everything you need to run your vLLM model. | ||
| registry. \<xx.yy\> is the version of Triton that you want to use. Please note, | ||
| that Triton's vLLM container was first published in 23.10 release, so any prior |
There was a problem hiding this comment.
I would get rid of the subordinate clause (, so any prior verson will not work.). It is redundant and can be unclear for future Triton versions.
README.md
Outdated
| in [build.py](https://github.com/triton-inference-server/server/blob/main/build.py) | ||
| for the Triton version you are using. | ||
|
|
||
| If you would like to use a specific vLLM commit or the latest version of vLLM, you |
There was a problem hiding this comment.
I think this is no longer necessary.
To get the latest vLLM, you can do: pip install git+https://github.com/vllm-project/vllm.git
To get a specific vLLM commit, you can do something like: pip install git+https://github.com/vllm-project/vllm.git@7d761fe3c12e87df37383467c43c97dec2bb8470
Co-authored-by: dyastremsky <[email protected]>
|
Great work! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Avoids a lot of confusion and improves user experience.