Move UMD tests after build artifact

[blozano-tt]

Ticket

NA

Problem description

UMD Tests were running independent of the rest of all post commit.
Many times I've seen UMD tests running, even if the global build of the project fails.
UMD Tests were getting built in the existing workflow on VMs that aren't really fast for building, and no ccache.
By putting UMD Tests after build-artifact we might be able to save on test machine usage.

** UMD TESTS CURRENTLY FAILING DUE TO MISSING FILE??? **

What's changed

UMD tests moved after build-artifact

Checklist

https://github.com/tenstorrent/tt-metal/actions/runs/11864188038

• Post commit CI passes
• Blackhole Post commit (if applicable)
• Model regression CI testing passes (if applicable)
• Device performance regression CI testing passes (if applicable)
• New/Existing tests provide coverage for changes

[blozano-tt]

Blocking Problem

• Invalid path may be embedded in test executable

I am having some problem with UMD tests.
If UMD is built on machine A. (Highly parallel builder machine with no Tenstorrent cards).
And I run tests on machine B. (build products transferred to arbitrary directory on machine B from machine A).
I end up hitting this error:

unknown file: Failure
C++ exception with description "Error: device descriptor file /home/ubuntu/actions-runner-2/_work/tt-metal/tt-metal/tt_metal/third_party/umd/tests/soc_descs/grayskull_10x12.yaml does not exist!" thrown in the test body.

The absolute path from the builder machine is baked into the test case! I don't have this path on machine B.
Running tests from the build tree is usually a problem like this, this is why there is the concept of installing tests, libraries, meta data files. Artifacts in an install tree are usually relocatable. i.e. they can be in /usr/lib or /opt/tenstorrent or what have you, and everything still works.
How can we make this better?
ENV that points to the yaml?
Embed the yaml info into the library?
Start running tests from the install tree?

[tt-rkim]

still draft?

[blozano-tt]

still draft?

If we ever want to optimize this (use build artifacts for UMD tests), a fix will be required first on UMD side.

I'm not going to prioritize this. For now, we'll just keep building on the VM.

[ttmchiou]

note that we've removed UMD tests on post-commit since UMD now has their own CI