tstesco/use-env-vars #96
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tstescoTT
added a commit
that referenced
this pull request
Feb 5, 2025
tstescoTT
added a commit
that referenced
this pull request
Feb 5, 2025
# change log
* Default handling of MESH_DEVICE for Llama 3.x models
* Modified setup script improvements:
* Improved environment variable handling and persistence storage integration
* Added IMPL_ID field (set to "tt-metal" for all current models)
* Introduced MODEL_VERSION and MODEL_ID variables for better versioning
* Add image input support for image-text-to-text models in client scripts and tools
* Added support for image input in trace capturing
* Added new parameters for image width and height
* Implemented handling of both text-only and image+text trace captures
* Rename client side scripts batch_size options to max_concurrent to indicate client side concurrent request limits
* Fixed the vLLM model registration logic. Added missing ModelRegistry.register_model call for TTLlamaForCausalLM for legacy implementation models
* Updated benchmark path handling to use $HOME environment variable instead of hardcoded /home/user path
* Add benchmark summary support handling for vllm benchmark script, add documentation example
* Added support for a new model "DeepSeek-R1-Distill-Llama-70B" in the model setup configurations
* use CACHE_ROOT and vllm_dir where possible, fix mock.vllm.openai.dockerfile (#96)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
addressing comments: