Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Adding Logging #33

Closed
wants to merge 78 commits into from
+396 −9
Closed

WIP: Adding Logging #33

Show file tree
Hide file tree
Changes from 73 commits
Commits
Show all changes
78 commits
Select commit Hold shift + click to select a range
36f5b77
added mock model
mvanniasingheTT Oct 22, 2024
ee9be29
adding vLLM dockerfile
tstescoTT Oct 15, 2024
efe648b
adding label GHCR and sed import edit for vllm server_example_tt.py
tstescoTT Oct 22, 2024
0e5f431
adding evals instructions and run script
tstescoTT Oct 22, 2024
ffdd77a
move vllm llama 3.1 70b implementation to top level model impl dir
tstescoTT Oct 23, 2024
5ef0d64
update eval instructions
tstescoTT Oct 23, 2024
33340dc
adding llama 3.1 70b benchmarking instructions
tstescoTT Oct 23, 2024
50626e4
adding dir for locust
tstescoTT Oct 23, 2024
11c604a
add doc link for vllm setup
tstescoTT Oct 23, 2024
45f544d
adding pre-commit with ruff linting and formatting
tstescoTT Oct 23, 2024
64281b0
add pre-commit instructions
tstescoTT Oct 23, 2024
f9b4fb3
add GHCR repo connection label to Dockerfile
tstescoTT Oct 23, 2024
3efd4c6
created + add mock model and mock offline inference w/ patches
mvanniasingheTT Oct 24, 2024
f2b8423
removed unneeded imports from mock model
mvanniasingheTT Oct 24, 2024
438e3ae
Merge branch 'tstesco/evals-benchmarking-structure' into mvanniasingh…
mvanniasingheTT Oct 24, 2024
b687ace
adding vLLM dockerfile
tstescoTT Oct 15, 2024
ed34e60
adding label GHCR and sed import edit for vllm server_example_tt.py
tstescoTT Oct 22, 2024
fd0c87d
adding evals instructions and run script
tstescoTT Oct 22, 2024
4d0cb6b
move vllm llama 3.1 70b implementation to top level model impl dir
tstescoTT Oct 23, 2024
a793d35
update eval instructions
tstescoTT Oct 23, 2024
7ed83b1
adding llama 3.1 70b benchmarking instructions
tstescoTT Oct 23, 2024
0e19de6
adding dir for locust
tstescoTT Oct 23, 2024
4f7a328
add doc link for vllm setup
tstescoTT Oct 23, 2024
b40c536
adding pre-commit with ruff linting and formatting
tstescoTT Oct 23, 2024
488c19c
add pre-commit instructions
tstescoTT Oct 23, 2024
ee888d9
add GHCR repo connection label to Dockerfile
tstescoTT Oct 23, 2024
5a0a124
update python path to include vllm
mvanniasingheTT Oct 24, 2024
6c35b17
adding requirements-dev.txt
tstescoTT Oct 25, 2024
022cd77
adding EOF newlines
tstescoTT Oct 25, 2024
f17a922
add ifeval to run_evals.sh, set to stream=False
tstescoTT Oct 25, 2024
3b48006
uncommetned sys path line
mvanniasingheTT Oct 25, 2024
c1b152c
update dockerfile to pip install for specifc commit of vllm
mvanniasingheTT Oct 25, 2024
291e872
update readme mock instructions
mvanniasingheTT Oct 25, 2024
ddbb7d4
update readme mock instructions
mvanniasingheTT Oct 25, 2024
443e930
pin dev dependency versions
tstescoTT Oct 25, 2024
40ba018
comments for mock patches
mvanniasingheTT Oct 25, 2024
ed15e1c
update vllm eval docs and development docs
tstescoTT Oct 25, 2024
f2b1f1b
update mock to work with latest vllm commit 82dbca6
mvanniasingheTT Oct 28, 2024
65ff822
remove tracing calls in mock model
mvanniasingheTT Oct 28, 2024
16320de
remove uneeded imports
mvanniasingheTT Oct 28, 2024
57a102c
remove checkout specfic commit
mvanniasingheTT Oct 28, 2024
01ee140
remove imports and comments
mvanniasingheTT Oct 28, 2024
7a92a47
cleanup
mvanniasingheTT Oct 28, 2024
42bd5a6
remove fork branch for tstescoTT/lm-evaluation-harness, can now use main
tstescoTT Oct 28, 2024
f2ac1cc
remove fork branch for tstescoTT/lm-evaluation-harness, can now use main
tstescoTT Oct 28, 2024
b7a9406
adding TT_VLLM_COMMIT_SHA_OR_TAG to Dockerfile
tstescoTT Oct 28, 2024
77cb4ba
uplift tt-metal and vllm commit SHA
tstescoTT Oct 28, 2024
7de2b00
adding TT_VLLM_COMMIT_DOCKER_TAG in vllm dev docker build instructions
tstescoTT Oct 30, 2024
f1ad0e1
adding comment header for ifeval
tstescoTT Oct 30, 2024
cbbc80e
fix comment
tstescoTT Oct 30, 2024
dc27981
update vllm commit sha
tstescoTT Oct 30, 2024
68ffdb4
adding logging
mvanniasingheTT Oct 30, 2024
43dff21
adding additonal tools, removing symlink for tt-metal models
tstescoTT Oct 30, 2024
213d7ef
update Dockerfile version in docs
tstescoTT Oct 31, 2024
fb12460
remove debug tools
tstescoTT Oct 31, 2024
bdd86a0
make logger point to same logger in vllm dir to that logger.info() works
mvanniasingheTT Oct 31, 2024
ce17c50
update tpot logging - NOTE: mult by num scheduler steps
mvanniasingheTT Oct 31, 2024
f46656e
resovled merge conflcits
mvanniasingheTT Nov 1, 2024
b2d8393
update to take num scheduler steps from engine args
mvanniasingheTT Nov 1, 2024
1b36a4e
Merge branch 'mvanniasinghe/logging' of https://github.com/tenstorren…
mvanniasingheTT Nov 1, 2024
fd873ab
resolve merge conflict
mvanniasingheTT Nov 6, 2024
5a7305c
resolve merge conflict + add logger_utils.py
mvanniasingheTT Nov 6, 2024
93aaf10
added online logging
mvanniasingheTT Nov 7, 2024
c9d1ec1
update MQLLMEngine to use scheduler_config to get num steps
mvanniasingheTT Nov 8, 2024
ceb0fea
update variable name rot_idxs_tt
mvanniasingheTT Nov 8, 2024
4c84da4
clean up imports
mvanniasingheTT Nov 12, 2024
c21a854
move new init over to logging utils + comments
mvanniasingheTT Nov 12, 2024
4e150a2
added jsonl logging - issue with online
mvanniasingheTT Nov 12, 2024
e3efb5b
remove breakpoint
mvanniasingheTT Nov 12, 2024
092fbbf
added line by line logging for offlien
mvanniasingheTT Nov 12, 2024
e09b428
updated loggin util with jsonl logger
mvanniasingheTT Nov 12, 2024
ea29384
added fix for when there is no decode (only generate first token/prefill
mvanniasingheTT Nov 13, 2024
1f9d587
fix for only decode + better logging online - batch size hard coded
mvanniasingheTT Nov 13, 2024
f9395b4
update to ignore .jsonl
mvanniasingheTT Nov 14, 2024
70472e7
delete data
mvanniasingheTT Nov 14, 2024
9dbc918
added .env.llama31_8b
mvanniasingheTT Nov 14, 2024
75471e8
update to use variable batch size - remove hard code
mvanniasingheTT Nov 14, 2024
49694c0
update logging utils with changes
mvanniasingheTT Nov 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading