Added new model support Cohere/Command-R #154
Conversation
quic-akuruvil
added
wip
| # | ||
| # Copyright (c) 2024 Qualcomm Innovation Center, Inc. All rights reserved. | ||
| # SPDX-License-Identifier: BSD-3-Clause | ||
| # |
There was a problem hiding this comment.
can you add a delimiter here?
|
@quic-akuruvil can you please add accuracy evaluation run details? |
Yes, I tried running the perplexity script, but it broke. I’ll look into it and get the numbers. |
| @@ -72,6 +72,7 @@ def main( | |||
| cache_dir=cache_dir, | |||
| hf_token=hf_token, | |||
| ) | |||
| embeds, config = get_embeddings(model_name, hf_token, cache_dir, local_model_dir) | |||
There was a problem hiding this comment.
Do we want to generalize this? I don't this is accessible for all the model categories.
There was a problem hiding this comment.
Yes right, we can make this conditional, fetch embeddings only for Cohere.
| @@ -204,18 +204,23 @@ def export_kvstyle_transformed_model_to_onnx( | |||
| raise ValueError(f"Need seq_len to be greater than zero, got seq_len={seq_len}") | |||
|
|
|||
| # Preprocess inputs | |||
| embeds = None | |||
| if model_name == "CohereForAI/c4ai-command-r-v01": | |||
There was a problem hiding this comment.
If we want to do this for all the versions of cohere and not specific to this model_name, then we need to do it as torch level? not a good practice to do at model_name level
There was a problem hiding this comment.
okay sure, I have changed it to those architecture with CohereCausalLM head in architecture, so it's more generic.
The perplexity is matching for onnx and qpc. |
for CL=1024, perplexity results are as below: |
Done |
|
@quic-akuruvil Can you please resolve the conflicts? |
Signed-off-by: Ann <[email protected]>
Signed-off-by: Ann <[email protected]>
quic-akuruvil
force-pushed
the
cohere
branch
from
61ca907 to
5eaf4bb
Compare
Signed-off-by: Ann <[email protected]>
Signed-off-by: Ann <[email protected]>
inputs_embedsinstead ofinput_idsto calculate embeddings in host.inputs_embeds