Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend support for the resize operator #1125

Open
wants to merge 4 commits into
base: dev
Choose a base branch
from

Conversation

klassen9
Copy link

@klassen9 klassen9 commented Jul 6, 2024

The PR extends/fixes the current implementations for the transformations of the resize operator in FINN.

We wanted to streamline a model that uses a resize operator, but the current implementation only supports resize via the scales input. The model we used, uses the sizes input. scales and sizes are both responsible for determining the amount of resizing and can be translated into each other. They are also mutually exclusive.

A problem that arose while implementing the support for sizes was to determine which input is used. Different versions of the Onnx opset have a different understanding of how to mark an input as unused. As an example:

  • Version 11 dictates: "If sizes is needed, the user must set scales to an empty tensor."
  • Version 13 dictates: "If sizes is needed, the user can use an empty string as the name of scales in this operator’s input list."

I am thus checking if scales or sizes exist by checking if they are None or are empty to support all versions. None is returned by model.get_initializer() if the input is an empty string in the input list.

Furthermore, this PR adds support for the Version 10 resize operation. The Version 10 resize operation only supports the scales input.

Other than the scales and sizes inputs and the input tensor, there is the roi (region of interest) input. The roi is not supported by the hardware implementation and therefore triggers an assert in convert_to_hw_layers.py when used.

The test_scale_resize_nhwc.py test was extended by a test that uses the sizes input instead of the scales input.

I also published a PR for qonnx adding the resize operation to the change_3d_tensors_to_4d transformation (fastmachinelearning/qonnx#125).

@klassen9
Copy link
Author

klassen9 commented Jul 7, 2024

Sorry, I didn't apply pre-commit correctly. Should be fixed in the second commit.

@klassen9 klassen9 marked this pull request as draft July 8, 2024 14:02
@klassen9 klassen9 marked this pull request as ready for review July 30, 2024 15:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant