Extend support for the resize operator #1125
Open
+141
−10
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The PR extends/fixes the current implementations for the transformations of the resize operator in FINN.
We wanted to streamline a model that uses a resize operator, but the current implementation only supports resize via the
scales
input. The model we used, uses thesizes
input.scales
andsizes
are both responsible for determining the amount of resizing and can be translated into each other. They are also mutually exclusive.A problem that arose while implementing the support for
sizes
was to determine which input is used. Different versions of the Onnx opset have a different understanding of how to mark an input as unused. As an example:sizes
is needed, the user must setscales
to an empty tensor."sizes
is needed, the user can use an empty string as the name ofscales
in this operator’s input list."I am thus checking if
scales
orsizes
exist by checking if they are None or are empty to support all versions. None is returned bymodel.get_initializer()
if the input is an empty string in the input list.Furthermore, this PR adds support for the Version 10 resize operation. The Version 10 resize operation only supports the
scales
input.Other than the
scales
andsizes
inputs and the input tensor, there is theroi
(region of interest) input. Theroi
is not supported by the hardware implementation and therefore triggers an assert inconvert_to_hw_layers.py
when used.The
test_scale_resize_nhwc.py
test was extended by a test that uses thesizes
input instead of thescales
input.I also published a PR for qonnx adding the resize operation to the change_3d_tensors_to_4d transformation (fastmachinelearning/qonnx#125).