Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

what is the use of preprocessing & postprossing ? can i start fastertransformer only for bloom model ? #168

Open
flyingjohn opened this issue Sep 22, 2023 · 1 comment
Labels
bug Something isn't working

Comments

@flyingjohn
Copy link

Description

i fail to start preprossing and postprossing. so i start fastertransformer only, it works fine but the model performance is bad. so i wonder if the reason is the missing of preprossing and postprossing ?? what is the use of preprocessing & postprossing ??

Reproduced Steps

/tritonserver/bin/tritonserver --model-repository=triton-model-store/bloom/
@flyingjohn flyingjohn added the bug Something isn't working label Sep 22, 2023
@yeahdongcn
Copy link

preprocessing: string -> encoded input_ids
bloom (fastertransformer): encoded input_ids -> encoded output_ids
postprocessing: encoded output_ids -> string

This could be fine if there is no need to run an end-to-end case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Development

No branches or pull requests

2 participants