Additional EDM fabric optimizations (mix of low level and experimental flow control protocol trimming)

[SeanNijjar]

Ticket

#17686

Problem description

EDM fabric perf is 💩 , make it better. We're SW bound. Put in known/obvious optimizations to make analysis less noisy.

What's changed

High level changes:

1. Optimize size information in packet header.

• simplifies packet processing and setup

2. Optimize routing information storage in packet header

• simplifies packet processing

3. Added missing inline write command type which is required after these changes
4. Migrate to more optimized eth APIs

• eth_write_reg and eth_send_packet that omit bit shifts and omit context switch calls

5. Trimming flow control protocols further

• various force inlines for tiny getter functions

Packet Header Size Field Optimization

• Simplify packet size storage and access
  • promote to "top-level" of packet to remove conditionality previously needed to get size info from packet
  • NOTE: packet size now specifies PAYLOAD SIZE ONLY!!! The header size must be implicitly added by fabric.
    • net this is still fine because we had to previous subtract header size when writing out to noc.

Packet Header Routing Info Optimization

Merged the mcast and unicast representation to match so I can uniformly process the packet to decide the following:

• Does packet get sent to local device noc?
• Does packet get forwarded through the fabric?

The previous implementation was required to first check the fabric send type before being able to do further inspection to answer the above questions. Now the code is much simpler - no fabric type info checked - single code path to check both. Additionally the check logic is also streamlined.

New packet command type

Extra functionality: Added NOC_UNICAST_INLINE_WRITE eth packet command type to address a regression as a result of the above change (if the command type wasn't added)

Optimized Eth send APIs

• Migrate eth_send_packet calls to new version that takes size in bytes.
  • This version avoids a number of shift operations that were present in the previously used version.
• Add and using new eth write remote reg (eth_write_remote_reg_no_txq_check) that doesn't have conditional context switch in body of function

Flow Control Protocol Trimming

• Enabled (by default) a less granular syncing mode between sender and receiver channels. Overall, in a theoretical sense, this is suboptimal. However, in a severely SW bound implementation like present, this will save on instruction count.

We disable the following:

• first level ack (i.e. when receiver gets the packet and notifies sender of packet received)
• separate pointer management for write flush ptr and completion pointer send on receiver channel
  • Flush ptr now merged with completion pointer so we cut down on processing.

Checklist

• All post commit CI: https://github.com/tenstorrent/tt-metal/actions/runs/13228444293
• T3K: https://github.com/tenstorrent/tt-metal/actions/runs/13228446457
  • Same fails as main
• TG: https://github.com/tenstorrent/tt-metal/actions/runs/13228449579
• Blackhole Post commit CI passes (if applicable)
• Model regression CI passes (if applicable)
• Device performance regression CI passes (if applicable)
• (For models and ops writers) Full new models tests CI passes (if applicable)
• New/Existing tests provide coverage for changes

[tt-aho]

lgtm

[TT-BrianLiu]

Files owned by me looks good