Code

Fixing trtllm OOM Issue#91

Code

Closed

KrishnanPrash wants to merge 15 commits intomaintriton-inference-server/triton_cli:mainfrom kprashanth-trtllm-fixtriton-inference-server/triton_cli:kprashanth-trtllm-fixCopy head branch name to clipboard

+222-1,449

Commits

Commits on Oct 25, 2024

• Switch to TRT-LLM LLM API over trtllm-build CLI workflow, clarify model support sections in README, add some better checks and error messages around --source arg, add 'local:' source prefix for par…

  Show description for c73d78d
  rmccorm4

  committed
  c73d78d

  Copy full SHA for c73d78d

• Bump to v0.1.0 and r24.09

  rmccorm4

  committed
  6b4597e

  Copy full SHA for 6b4597e

• Update grpcio version for 24.09 client and group the dependencies more closely

  rmccorm4

  committed
  960bd76

  Copy full SHA for 960bd76

• Disable triton profile tests due to genai-perf bug in 24.09 - should be restored in 24.10

  rmccorm4

  committed
  febed9b

  Copy full SHA for febed9b

Commits on Oct 30, 2024

• Try fixing TRTLLM tests running OOM across tests

  rmccorm4

  committed
  6672fec

  Copy full SHA for 6672fec

Commits on Nov 4, 2024

• [DEBUG] Catch signals in subprocess, and use pytest fixture to try to ensure server is cleaned properly between tests

  rmccorm4

  committed
  1914c71

  Copy full SHA for 1914c71

• [DEBUG] Add fixtures in conftest

  rmccorm4

  committed
  229fd0b

  Copy full SHA for 229fd0b

Commits on Nov 13, 2024

• Erroring out of nested subprocess explicitly

  KrishnanPrash

  committed
  7eb47c1

  Copy full SHA for 7eb47c1

• Cleaning up pt2

  KrishnanPrash

  committed
  805ab2c

  Copy full SHA for 805ab2c

• Adding WAR for genai-perf

  KrishnanPrash

  committed
  d31b18e

  Copy full SHA for d31b18e

Commits on Nov 15, 2024

• testing theories

  KrishnanPrash

  committed
  08c56a0

  Copy full SHA for 08c56a0

Commits on Nov 18, 2024

• testing changes

  KrishnanPrash

  committed
  1785711

  Copy full SHA for 1785711

• Cleaning up

  KrishnanPrash

  committed
  eefd3d9

  Copy full SHA for eefd3d9

• Cleaning up testing

  KrishnanPrash

  committed
  55f5184

  Copy full SHA for 55f5184

• Formatting and Cleaning up

  KrishnanPrash

  committed
  067bf67

  Copy full SHA for 067bf67