Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Web] FP8 is not supported #23241

Open
lsb opened this issue Jan 2, 2025 · 3 comments
Open

[Web] FP8 is not supported #23241

lsb opened this issue Jan 2, 2025 · 3 comments
Labels
platform:web issues related to ONNX Runtime web; typically submitted using template

Comments

@lsb
Copy link

lsb commented Jan 2, 2025

Describe the issue

When trying to create an inference session with a minimal model that casts input from fp32 into fp8 and back again, the wasm._OrtCreateSession fails with

Can't create a session. ERROR_CODE: 10, ERROR_MESSAGE: This is an invalid model. Type Error: Type 'tensor(float8e4m3fn)' of input parameter (_to_copy) of operator (Neg) in node (node_Neg_1) is invalid.

negf8.html

To reproduce

Running the attached html file (uploaded as markdown) will attempt to create an inference session with a model that uses FP8 precision for casting.

Urgency

No response

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

v1.20.1

Execution Provider

'wasm'/'cpu' (WebAssembly CPU)

@lsb lsb added the platform:web issues related to ONNX Runtime web; typically submitted using template label Jan 2, 2025
@lsb
Copy link
Author

lsb commented Jan 3, 2025

But this works fine in the newest Python runtime, for example

@fs-eire
Copy link
Contributor

fs-eire commented Jan 7, 2025

I am not sure why ONNX Runtime Python binding can run this model.

According to https://github.com/onnx/onnx/blob/main/docs/Operators.md#Neg, the operator Neg does not support type Float8E4M3FN, which is reported in the error message.

@fs-eire
Copy link
Contributor

fs-eire commented Jan 7, 2025

I tried the same model in python binding and it says the same error:

onnxruntime.capi.onnxruntime_pybind11_state.InvalidGraph: [ONNXRuntimeError] : 10 : INVALID_GRAPH : Load model from c:/code/o2/js/web/a.onnx failed:This is an invalid model. Type Error: Type 'tensor(float8e4m3fn)' of input parameter (_to_copy) of operator (Neg) in node (node_Neg_1) is invalid.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
platform:web issues related to ONNX Runtime web; typically submitted using template
Projects
None yet
Development

No branches or pull requests

2 participants