Skip to content

Commit

Permalink
[TDX] Increased VLLM memory resources
Browse files Browse the repository at this point in the history
Signed-off-by: Jakub Ledworowski <[email protected]>
  • Loading branch information
JakubLedworowski committed Feb 12, 2025
1 parent 7b6e136 commit 8b39494
Showing 1 changed file with 2 additions and 4 deletions.
6 changes: 2 additions & 4 deletions helm-charts/TDX.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,13 +87,11 @@ Follow the steps below to deploy ChatQnA:
--set retriever-usvc.tdxEnabled=true --set retriever-usvc.resources.limits.memory=8Gi \
--set tei.tdxEnabled=true --set tei.resources.limits.memory=8Gi \
--set teirerank.tdxEnabled=true --set teirerank.resources.limits.memory=8Gi \
--set vllm.tdxEnabled=true --set vllm.resources.limits.memory=48Gi
--set vllm.tdxEnabled=true --set vllm.resources.limits.memory=80Gi
```

> [!NOTE]
> The `resources.limits` and `resources.requests` needs to be set when the Intel TDX is used.
>
> The above example sets the memory limits and requests to 4Gi for each microservice.
> The `resources.limits` needs to be set when the Intel TDX is used.
>
> By default, each Kubernetes pod will be assigned `1` CPU and `2Gi` of memory, but half of it will be used for filesystem.
>
Expand Down

0 comments on commit 8b39494

Please sign in to comment.