__setitem__ for tensors

[afuller-TT]

From the community:

Lack of __setitem__ for tensors. I had to emulate it and it isn't pretty.

[TT-billteng]

why would this issue be this team's responsiblity?

[marty1885]

For context, this need comes from my GGML backend where GGML's unittests tests for the ability to write back into tensors generally (as a part of supporting KV Cache). Currently the only way to do this is

1. If it happens to be writing a row in a 3D tensor - use the KV cache update method as it's a fast path
2. Else, slice and dice the original tensor, Concat 4 pieces from the original 1 from the updated data. To form a new tensor

Path 2 is extremely slow and ugly. It'll be much better if there's native support. It's something that's bugging me almost since day 1 so it's on my list of stuff when people asked what to improve.

[TT-billteng]

For context, this need comes from my GGML backend where GGML's unittests tests for the ability to write back into tensors generally (as a part of supporting KV Cache). Currently the only way to do this is

1. If it happens to be writing a row in a 3D tensor - use the KV cache update method as it's a fast path
2. Else, slice and dice the original tensor, Concat 4 pieces from the original 1 from the updated data. To form a new tensor

Path 2 is extremely slow and ugly. It'll be much better if there's native support. It's something that's bugging me almost since day 1 so it's on my list of stuff when people asked what to improve.

By "this team" I'm referring to software infrastructure, not metal project as a whole. We should either re-assign to metal runtime or TTNN.

[afuller-TT]

@TT-billteng I was just trying to record the feedback we received so it didn't get lost. And figured we could figure out the right team afterwards.

[TT-billteng]

@ayerofieiev-tt please re-assign to the right board if necessary