Convert `aten.ceil` to `ttnn.ceil` #198

jdh8 · 2024-09-13T19:30:17Z

Ticket

Working around Some pointwise unary ops return 2-D results from 1-D inputs tt-metal#12671
Resolves aten.ceil.default #493

Problem description

Convert aten.ceil to ttnn.ceil and probably other rounding ops

Currently, 1-D cases fail because ttnn.ceil produces 2-D results. For instance, ttnn.ceil takes a (1066,) tensor but produces a (1, 1066) tensor.

FAILED tests/lowering/eltwise/unary/test_ceil.py::test_ceil[input_shape0] - AssertionError: list(expected_pytorch_result.shape)=[1066] vs list(actual_pytorch_result.shape)=[1, 1066]
FAILED tests/lowering/eltwise/unary/test_ceil.py::test_ceil[input_shape1] - AssertionError: list(expected_pytorch_result.shape)=[120] vs list(actual_pytorch_result.shape)=[1, 120]
FAILED tests/lowering/eltwise/unary/test_ceil.py::test_ceil[input_shape2] - AssertionError: list(expected_pytorch_result.shape)=[128] vs list(actual_pytorch_result.shape)=[1, 128]
FAILED tests/lowering/eltwise/unary/test_ceil.py::test_ceil[input_shape3] - AssertionError: list(expected_pytorch_result.shape)=[160] vs list(actual_pytorch_result.shape)=[1, 160]

I doubt if this happens to other elementwise ops, too.

What's changed

Convert aten.ceil to ttnn.ceil
Test conversion of aten.ceil
Convert aten.round to ttnn.round
Test conversion of aten.round
Convert aten.trunc to ttnn.trunc
Test conversion of aten.trunc

ayerofieiev-tt · 2024-09-13T19:39:50Z

@jdh8 , lets fire a ticket and lets overcome in compiler by squeezing the dimension?

jdh8 · 2024-09-13T20:20:18Z

I'll probably deal with this PR after #113 and #170. I suggest patching once for all univariate functions because there are many of them.

jdh8 · 2024-10-03T17:10:09Z

Unfortunately, the workaround by squeezing out the extraneous dimension for 1-D tensors (e6e9ef8) still leaves errors. It cannot resolve tenstorrent/tt-metal#12671.

FAILED tests/lowering/eltwise/unary/test_ceil.py::test_ceil[input_shape0] - RuntimeError: TT_FATAL @ ../ttnn/cpp/ttnn/tensor/types.cpp:170: normalized_index >= 0 and normalized_index < rank
FAILED tests/lowering/eltwise/unary/test_ceil.py::test_ceil[input_shape1] - RuntimeError: TT_FATAL @ ../ttnn/cpp/ttnn/tensor/types.cpp:170: normalized_index >= 0 and normalized_index < rank
FAILED tests/lowering/eltwise/unary/test_ceil.py::test_ceil[input_shape2] - RuntimeError: TT_FATAL @ ../ttnn/cpp/ttnn/tensor/types.cpp:170: normalized_index >= 0 and normalized_index < rank
FAILED tests/lowering/eltwise/unary/test_ceil.py::test_ceil[input_shape3] - RuntimeError: TT_FATAL @ ../ttnn/cpp/ttnn/tensor/types.cpp:170: normalized_index >= 0 and normalized_index < rank
FAILED tests/lowering/eltwise/unary/test_cos.py::test_cos[input_shape1] - RuntimeError: TT_FATAL @ ../ttnn/cpp/ttnn/tensor/types.cpp:170: normalized_index >= 0 and normalized_index < rank
FAILED tests/lowering/eltwise/unary/test_erf.py::test_erf[input_shape1] - RuntimeError: TT_FATAL @ ../ttnn/cpp/ttnn/tensor/types.cpp:170: normalized_index >= 0 and normalized_index < rank
FAILED tests/lowering/eltwise/unary/test_exp.py::test_exp[input_shape1] - RuntimeError: TT_FATAL @ ../ttnn/cpp/ttnn/tensor/types.cpp:170: normalized_index >= 0 and normalized_index < rank
FAILED tests/lowering/eltwise/unary/test_floor.py::test_floor[input_shape4] - RuntimeError: TT_FATAL @ ../ttnn/cpp/ttnn/tensor/types.cpp:170: normalized_index >= 0 and normalized_index < rank
FAILED tests/lowering/eltwise/unary/test_gelu.py::test_gelu[input_shape1] - RuntimeError: TT_FATAL @ ../ttnn/cpp/ttnn/tensor/types.cpp:170: normalized_index >= 0 and normalized_index < rank
FAILED tests/lowering/eltwise/unary/test_neg.py::test_neg[input_shapes0] - RuntimeError: TT_FATAL @ ../ttnn/cpp/ttnn/tensor/types.cpp:170: normalized_index >= 0 and normalized_index < rank
FAILED tests/lowering/eltwise/unary/test_remainder.py::test_remainder[input_shape0-7-True] - AssertionError: assert 0 == 1
FAILED tests/lowering/eltwise/unary/test_remainder.py::test_remainder[input_shape1-3-True] - AssertionError: assert 0 == 1
FAILED tests/lowering/eltwise/unary/test_remainder.py::test_remainder[input_shape2-5-True] - AssertionError: assert 0 == 1
FAILED tests/lowering/eltwise/unary/test_remainder.py::test_remainder[input_shape3-1-True] - AssertionError: assert 0 == 1
FAILED tests/lowering/eltwise/unary/test_remainder.py::test_remainder[input_shape4-1-True] - AssertionError: assert 0 == 1
FAILED tests/lowering/eltwise/unary/test_rsqrt.py::test_rsqrt[input_shape1] - RuntimeError: TT_FATAL @ ../ttnn/cpp/ttnn/tensor/types.cpp:170: normalized_index >= 0 and normalized_index < rank
FAILED tests/lowering/eltwise/unary/test_sqrt.py::test_sqrt[input_shape1] - RuntimeError: TT_FATAL @ ../ttnn/cpp/ttnn/tensor/types.cpp:170: normalized_index >= 0 and normalized_index < rank
========================================================================= 17 failed, 43 passed in 39.79s ==========================================================================

jerrysky3 · 2024-10-04T02:28:40Z

Unfortunately, the workaround by squeezing out the extraneous dimension for 1-D tensors (e6e9ef8) still leaves errors. It cannot resolve tenstorrent/tt-metal#12671.

FAILED tests/lowering/eltwise/unary/test_ceil.py::test_ceil[input_shape0] - RuntimeError: TT_FATAL @ ../ttnn/cpp/ttnn/tensor/types.cpp:170: normalized_index >= 0 and normalized_index < rank
FAILED tests/lowering/eltwise/unary/test_ceil.py::test_ceil[input_shape1] - RuntimeError: TT_FATAL @ ../ttnn/cpp/ttnn/tensor/types.cpp:170: normalized_index >= 0 and normalized_index < rank
FAILED tests/lowering/eltwise/unary/test_ceil.py::test_ceil[input_shape2] - RuntimeError: TT_FATAL @ ../ttnn/cpp/ttnn/tensor/types.cpp:170: normalized_index >= 0 and normalized_index < rank
FAILED tests/lowering/eltwise/unary/test_ceil.py::test_ceil[input_shape3] - RuntimeError: TT_FATAL @ ../ttnn/cpp/ttnn/tensor/types.cpp:170: normalized_index >= 0 and normalized_index < rank
FAILED tests/lowering/eltwise/unary/test_cos.py::test_cos[input_shape1] - RuntimeError: TT_FATAL @ ../ttnn/cpp/ttnn/tensor/types.cpp:170: normalized_index >= 0 and normalized_index < rank
FAILED tests/lowering/eltwise/unary/test_erf.py::test_erf[input_shape1] - RuntimeError: TT_FATAL @ ../ttnn/cpp/ttnn/tensor/types.cpp:170: normalized_index >= 0 and normalized_index < rank
FAILED tests/lowering/eltwise/unary/test_exp.py::test_exp[input_shape1] - RuntimeError: TT_FATAL @ ../ttnn/cpp/ttnn/tensor/types.cpp:170: normalized_index >= 0 and normalized_index < rank
FAILED tests/lowering/eltwise/unary/test_floor.py::test_floor[input_shape4] - RuntimeError: TT_FATAL @ ../ttnn/cpp/ttnn/tensor/types.cpp:170: normalized_index >= 0 and normalized_index < rank
FAILED tests/lowering/eltwise/unary/test_gelu.py::test_gelu[input_shape1] - RuntimeError: TT_FATAL @ ../ttnn/cpp/ttnn/tensor/types.cpp:170: normalized_index >= 0 and normalized_index < rank
FAILED tests/lowering/eltwise/unary/test_neg.py::test_neg[input_shapes0] - RuntimeError: TT_FATAL @ ../ttnn/cpp/ttnn/tensor/types.cpp:170: normalized_index >= 0 and normalized_index < rank
FAILED tests/lowering/eltwise/unary/test_remainder.py::test_remainder[input_shape0-7-True] - AssertionError: assert 0 == 1
FAILED tests/lowering/eltwise/unary/test_remainder.py::test_remainder[input_shape1-3-True] - AssertionError: assert 0 == 1
FAILED tests/lowering/eltwise/unary/test_remainder.py::test_remainder[input_shape2-5-True] - AssertionError: assert 0 == 1
FAILED tests/lowering/eltwise/unary/test_remainder.py::test_remainder[input_shape3-1-True] - AssertionError: assert 0 == 1
FAILED tests/lowering/eltwise/unary/test_remainder.py::test_remainder[input_shape4-1-True] - AssertionError: assert 0 == 1
FAILED tests/lowering/eltwise/unary/test_rsqrt.py::test_rsqrt[input_shape1] - RuntimeError: TT_FATAL @ ../ttnn/cpp/ttnn/tensor/types.cpp:170: normalized_index >= 0 and normalized_index < rank
FAILED tests/lowering/eltwise/unary/test_sqrt.py::test_sqrt[input_shape1] - RuntimeError: TT_FATAL @ ../ttnn/cpp/ttnn/tensor/types.cpp:170: normalized_index >= 0 and normalized_index < rank
========================================================================= 17 failed, 43 passed in 39.79s ==========================================================================

I suspect the problem is the tensor returned by ttnn.ceil is in TILE_LAYOUT, which ttnn.squeeze don't know how to turn an 2D tiled tensor into 1d. It works for me if the tensor is first converted into ROW_MARJO_LAYOUT:

result_after = ttnn.ceil(input_tensor)
result_after = ttnn.to_layout(result_after, ttnn.ROW_MAJOR_LAYOUT)
result_after = ttnn.squeeze(result_after, 0)

jdh8 · 2024-10-24T16:21:51Z

My test results:

(python_env) jdh8@tt-loudbox:~/pytorch2.0_ttnn$ pytest tests/lowering/eltwise/unary/test_round.py 
=============================== test session starts ================================
platform linux -- Python 3.8.10, pytest-7.2.2, pluggy-1.5.0
rootdir: /home/jdh8/pytorch2.0_ttnn/tests, configfile: pytest.ini
plugins: split-0.8.2, anyio-4.5.2, xdist-3.6.1, dash-2.15.0, timeout-2.2.0
collected 5 items                                                                  

tests/lowering/eltwise/unary/test_round.py .....                             [100%]

================================ 5 passed in 3.40s =================================
                 Device | INFO     | Closing user mode device drivers

jdh8 · 2024-11-23T16:33:58Z

All the failing ops are aten.max_pool2d_with_indices_backward.default from U-Net and friends. We can catch these ops with

pytest tests/autogen_op/U*-train/*aten_max_pool2d_with_indices_backward_default.py

jdh8 · 2024-11-26T01:51:50Z

Workaround issues left:

jdh8 · 2024-11-26T20:41:06Z

Tests are running at https://github.com/tenstorrent/pytorch2.0_ttnn/actions/runs/12038678310

ayerofieiev-tt · 2024-11-26T22:27:54Z

torch_ttnn/passes/lowering/to_tt_pass.py

+                return unsqueeze_to_2d(TTNN_POINTWISE_UNARY_OPS[node.target])
+
+            if node.target == torch.ops.aten.round.default:
+                return unsqueeze_to_2d(ttnn.round, (args[0],), {"decimals": 0})


Must update ttnn op binding to default the argument to 0?

That would be great! I once tried at tenstorrent/tt-metal#13851 but in vain.

ayerofieiev-tt · 2024-11-26T22:29:13Z

torch_ttnn/passes/lowering/to_tt_pass.py

+                return result
+
+            if node.target in TTNN_POINTWISE_UNARY_OPS:
+                return unsqueeze_to_2d(TTNN_POINTWISE_UNARY_OPS[node.target])


What do we unsqueeze here? Why only for unary?

Thanks for inspiring me with a simpler algorithm. Reshape the result back when ndims < 2.

jdh8 · 2024-11-29T11:47:33Z

I cannot locally reproduce the errors found in CI.
https://github.com/tenstorrent/pytorch2.0_ttnn/actions/runs/12073595035/job/33670553655#step:5:26198

==== 4 passed, 156 deselected, 2 xfailed, 8 warnings in 1076.48s (0:17:56) =====
|                 path                 |     passed     |   xfailed    | subtotal |
| ------------------------------------ | -------------- | ------------ | -------: |
| models/falcon/test_falcon.py         | :green_circle: |              |        1 |
| models/flan_t5/test_flan_t5.py       | :green_circle: | :red_circle: |        2 |
| models/glpn_kitti/test_glpn_kitti.py | :green_circle: | :red_circle: |        2 |
| models/gpt2/test_gpt2.py             | :green_circle: |              |        1 |
| TOTAL                                |              4 |            2 |        6 |

However, ResNet raises these errors only locally.

ERROR tests/models/resnet/test_resnet.py::test_resnet[train] - TypeError: ResNet18-train compiled failed to run.
ERROR tests/models/resnet/test_resnet.py::test_resnet[eval] - TypeError: ResNet18 compiled failed to run.

ayerofieiev-tt · 2024-11-29T17:14:59Z

@kevinwuTT if you can suggest anything to enable debugging of this case

The results are inconclusive: - `cos` passes - `sqrt` and `floor` fail

jdh8 · 2024-12-19T22:23:20Z

Tests finally pass!
https://github.com/tenstorrent/pytorch2.0_ttnn/actions/runs/12420284637

ayerofieiev-tt · 2024-12-19T22:49:51Z

Tests pass https://github.com/tenstorrent/pytorch2.0_ttnn/actions/runs/12420284637

jdh8 added blocked conversion labels Sep 13, 2024

jdh8 requested review from kevinwuTT and ayerofieiev-tt September 13, 2024 19:30

jdh8 self-assigned this Sep 13, 2024

jdh8 mentioned this pull request Sep 13, 2024

Some pointwise unary ops return 2-D results from 1-D inputs tenstorrent/tt-metal#12671

Open

jdh8 force-pushed the feature/rounding branch from bc4e92b to 1afebf9 Compare September 25, 2024 15:05

jdh8 force-pushed the feature/rounding branch from 1dec51a to 2f75398 Compare October 3, 2024 16:29

jdh8 force-pushed the feature/rounding branch from 56af042 to 1845ac2 Compare October 13, 2024 15:21

jdh8 removed the blocked label Oct 13, 2024

jdh8 force-pushed the feature/rounding branch from 1845ac2 to c64f168 Compare October 16, 2024 12:47

jdh8 mentioned this pull request Oct 16, 2024

#13385: Direct kernel function for ttnn.round tenstorrent/tt-metal#13851

Open

8 tasks

jdh8 force-pushed the feature/rounding branch from c64f168 to a62fc8e Compare October 24, 2024 10:44

jdh8 mentioned this pull request Nov 22, 2024

Convert aten.hardsigmoid to ttnn.hardsigmoid #491

Merged

4 tasks

jdh8 added enhancement New feature or request and removed enhancement New feature or request labels Nov 22, 2024

jdh8 force-pushed the feature/rounding branch from a5c1d24 to 408f6ba Compare November 23, 2024 02:14

jdh8 force-pushed the feature/rounding branch from d2de299 to 2c17e8d Compare November 25, 2024 15:02

jdh8 force-pushed the feature/rounding branch 2 times, most recently from 204dfe0 to 4c07949 Compare November 26, 2024 20:00

ayerofieiev-tt reviewed Nov 26, 2024

View reviewed changes

jdh8 force-pushed the feature/rounding branch 2 times, most recently from fe9f57c to 59c0fb2 Compare December 12, 2024 21:56

jdh8 added 19 commits December 19, 2024 15:10

Test conversion to ttnn.floor with known input variations

6ca70a2

Make import global to reduce overhead

98b0c36

Convert aten.ceil to ttnn.ceil

2b2cb01

Try (1066,) with some other univariate functions

ff3e561

The results are inconclusive: - `cos` passes - `sqrt` and `floor` fail

Test unary ops with fast and approximate mode

b38106b

Try squeezing out the extraneous dimension for 1-D tensors

f90e621

Restore conversion for aten.remainder.Scalar

ebbdf06

Fix the workaround to squeeze back to 1-D tensors

a4848ce

Implement conversion to ttnn.round

bfec5ca

Implement conversion to ttnn.trunc

a5912f0

Update conversion for ttnn.round

6967115

Lessen PCC for ttnn.round kernel (tenstorrent/tt-metal#13851)

5db5524

Convert torch.round with various decimal places

d62513d

Update parameters of ttnn.round

e7fea23

Restore code wrongly removed when rebasing 1355625

da7d927

Update list of pointwise unary ops

daf2d51

Restore general handler for aten.hardtanh

12a7129

Properly test 1-D cases after working around tenstorrent/tt-metal#12671

3a04e8c

Simplify the workaround for tenstorrent/tt-metal#12671

09535cd

jdh8 force-pushed the feature/rounding branch from 59c0fb2 to 09535cd Compare December 19, 2024 15:14

ayerofieiev-tt approved these changes Dec 19, 2024

View reviewed changes

Merge branch 'main' into feature/rounding

ae2d156

ayerofieiev-tt merged commit 75ef405 into main Dec 19, 2024
1 check passed

ayerofieiev-tt deleted the feature/rounding branch December 19, 2024 22:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Convert `aten.ceil` to `ttnn.ceil` #198

Convert `aten.ceil` to `ttnn.ceil` #198

jdh8 commented Sep 13, 2024 •

edited

Loading

ayerofieiev-tt commented Sep 13, 2024

jdh8 commented Sep 13, 2024

jdh8 commented Oct 3, 2024

jerrysky3 commented Oct 4, 2024 •

edited

Loading

jdh8 commented Oct 24, 2024

jdh8 commented Nov 23, 2024

jdh8 commented Nov 26, 2024

jdh8 commented Nov 26, 2024

ayerofieiev-tt Nov 26, 2024

jdh8 Nov 27, 2024

ayerofieiev-tt Nov 26, 2024

jdh8 Nov 27, 2024

jdh8 commented Nov 29, 2024

ayerofieiev-tt commented Nov 29, 2024

jdh8 commented Dec 19, 2024

ayerofieiev-tt commented Dec 19, 2024

Convert aten.ceil to ttnn.ceil #198

Convert aten.ceil to ttnn.ceil #198

Conversation

jdh8 commented Sep 13, 2024 • edited Loading

Ticket

Problem description

What's changed

ayerofieiev-tt commented Sep 13, 2024

jdh8 commented Sep 13, 2024

jdh8 commented Oct 3, 2024

jerrysky3 commented Oct 4, 2024 • edited Loading

jdh8 commented Oct 24, 2024

jdh8 commented Nov 23, 2024

jdh8 commented Nov 26, 2024

jdh8 commented Nov 26, 2024

ayerofieiev-tt Nov 26, 2024

Choose a reason for hiding this comment

jdh8 Nov 27, 2024

Choose a reason for hiding this comment

ayerofieiev-tt Nov 26, 2024

Choose a reason for hiding this comment

jdh8 Nov 27, 2024

Choose a reason for hiding this comment

jdh8 commented Nov 29, 2024

ayerofieiev-tt commented Nov 29, 2024

jdh8 commented Dec 19, 2024

ayerofieiev-tt commented Dec 19, 2024

Convert `aten.ceil` to `ttnn.ceil` #198

Convert `aten.ceil` to `ttnn.ceil` #198

jdh8 commented Sep 13, 2024 •

edited

Loading

jerrysky3 commented Oct 4, 2024 •

edited

Loading