Skip to content

Commit

Permalink
Update README.md (#69)
Browse files Browse the repository at this point in the history
* Update README.md

Point to previous config.pbtxt while the changes are not released in the container

* Update Quick_Deploy/vLLM/README.md

Use the release number instead of a commit hash for better readability

Co-authored-by: Olga Andreeva <[email protected]>

* Update README.md

point to release instead of main branch and add clarification suggested by @oandreeva-nv

---------

Co-authored-by: Olga Andreeva <[email protected]>
  • Loading branch information
joaquincabezas and oandreeva-nv authored Dec 28, 2023
1 parent d7521fe commit 336a7d7
Showing 1 changed file with 3 additions and 2 deletions.
5 changes: 3 additions & 2 deletions Quick_Deploy/vLLM/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,9 +54,10 @@ and
required to serve the [facebook/opt-125m](https://huggingface.co/facebook/opt-125m) model.
```
mkdir -p model_repository/vllm_model/1
wget -P model_repository/vllm_model/1 https://raw.githubusercontent.com/triton-inference-server/vllm_backend/main/samples/model_repository/vllm_model/1/model.json
wget -P model_repository/vllm_model/ https://raw.githubusercontent.com/triton-inference-server/vllm_backend/main/samples/model_repository/vllm_model/config.pbtxt
wget -P model_repository/vllm_model/1 https://raw.githubusercontent.com/triton-inference-server/vllm_backend/r<xx.yy>/samples/model_repository/vllm_model/1/model.json
wget -P model_repository/vllm_model/ https://raw.githubusercontent.com/triton-inference-server/vllm_backend/r<xx.yy>/samples/model_repository/vllm_model/config.pbtxt
```
where <xx.yy> is the version of Triton that you want to use. Please note, that Triton's vLLM container has been introduced starting from 23.10 release.

The model repository should look like this:
```
Expand Down

0 comments on commit 336a7d7

Please sign in to comment.