Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#5605: Only force-stall ethernet programs on earlier ethernet programs #16202

Merged
merged 1 commit into from
Dec 20, 2024

Conversation

jbaumanTT
Copy link
Contributor

Ticket

#15605

Problem description

On llama, we're seeing high dispatch latencies on programs that execute on active ethernet cores. This is because we always stall writing program binaries and launch messages until the previous program completes, since we don't support having program binaries in a ring buffer on ethernet cores.

What's changed

Keep track of when the last program using active ethernet cores was dispatched, so we can wait on that program before sending out binaries. This is better than always waiting on the immediate previous program, since in most cases we don't run programs on the ethernet cores back-to-back.

Checklist

  • Post commit CI passes
  • Blackhole Post commit (if applicable)
  • Model regression CI testing passes (if applicable)
  • Device performance regression CI testing passes (if applicable)
  • (For models and ops writers) Full new models tests passes
  • New/Existing tests provide coverage for changes

@jbaumanTT jbaumanTT force-pushed the jbauman/improveethernet branch 3 times, most recently from 41e89cc to 26b75a4 Compare December 20, 2024 06:20
Keep track of when the last program using active ethernet cores was dispatched,
so we can wait on that program before sending out binaries. This is better than
always waiting on the immediate previous program, since in most cases we don't
run programs on the ethernet cores back-to-back.
@jbaumanTT jbaumanTT changed the title #0: Only force-stall ethernet programs on earlier ethernet programs #5605: Only force-stall ethernet programs on earlier ethernet programs Dec 20, 2024
@jbaumanTT jbaumanTT force-pushed the jbauman/improveethernet branch from 26b75a4 to ef4a74e Compare December 20, 2024 17:02
@jbaumanTT jbaumanTT merged commit 5058b8f into main Dec 20, 2024
183 of 188 checks passed
@jbaumanTT jbaumanTT deleted the jbauman/improveethernet branch December 20, 2024 21:14
@blozano-tt blozano-tt restored the jbauman/improveethernet branch December 22, 2024 07:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants