Load model failed - error: Worker died #3104
Comments
geraldstanje
changed the title
|
Hi @geraldstanje Not too familiar with Sagemaker itself but the inference.py script you're providing looks like you're trying to deploy a setfit model. You will need to integrate this into a TorchServe handler and package it with the model-archiver into a tar.gz file which will add important meta information. Please have a look as our XGBoost example which should be easily adaptable to your use case as you can basically deploy any framework or library through this approach. Let me know if you have further questions. |
|
@mreso thanks for pointing out - is there a simple way to convertit to run setFit models with TorchServe? can i copy the code i have into a BaseHandler and implement those functions? does the sagemaker return the same datatype / format as the BaseHandler, what is required? |
|
cc @namannandan |
|
@geraldstanje yes, you basically follow the XGBoost example to create your own handler or if your model is a HuggingFace model from their transformers library you can just follow one of of these examples: Let me know if you're having problems converting your example. |
|
@mreso thanks - how is sagemaker than be able to use torchServe if they dont implement the ts.torch_handler.base_handler? lets say i take this as an example: https://github.com/aws/amazon-sagemaker-examples/blob/main/frameworks/pytorch/get_started_mnist_deploy.ipynb i looked at https://github.com/pytorch/serve/tree/master/examples/xgboost_classfication. how can i create the model.pt for the torch-model-archiver? |
|
@geraldstanje to answer your question
The PyTorch inference containers that are compatible with SageMaker install a package called the SageMaker PyTorch Inference Toolkit which provides a handler implementation that is compatible with TorchServe and plugs in the If you'd like to create a custom docker container that is SageMaker compatible, I would suggest starting out with a SageMaker PyTorch Inference Container as the base image and build on top of it. For ex: If you would like to use TorchServe natively on SageMaker, here's an example on the same: https://github.com/aws/amazon-sagemaker-examples/blob/main/inference/torchserve/mme-gpu/torchserve_multi_model_endpoint.ipynb Also, looking at the error logs, I see from the traceback that the model load failed because the handler was unable to find a necessary module: Could you please check if all the required dependencies to load the model are either installed in the container or included in the model archive? |
🐛 Describe the bug
Load model failed: policy_vs_doc_model_tar_gz, error: Worker died.
Error logs
Installation instructions
Dockerfile:
build and run docker:
Model Packaing
whats inside the policy_vs_doc_model.tar.gz:
tree view (info: extracted policy_vs_doc_model.tar.gz to model dir so we can tree view it):
./code/inference.py:
./code/requirements.txt:
config.properties
Versions
used docker image: pytorch/torchserve:latest-gpu
Repro instructions
Possible Solution
The text was updated successfully, but these errors were encountered: