Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Do not await for responses when recovering pub filters #2265

Merged
merged 5 commits into from
Jan 29, 2025

Conversation

msfstef
Copy link
Contributor

@msfstef msfstef commented Jan 29, 2025

The recover_shape and refresh_publication are in the init path of ShapeCache, and there is no reason why we would be waiting for responses.

The recover_shape has no expected failure as it simply updates a map with appropriate counters.

The refresh_publication will schedule a publication update, but we don't have to wait for it to finish to consider ShapeCache ready, as the shape consumers are already up and running and any new shapes will trigger another publication update after this one.

These changes are based off of Sentry errors noticed from the cloud.

I've also set the publication update debounce time to 0ms, so now it simply waits until end of current process message queue. This addresses an unnecessary delay in prepare_table calls (@robacourt), since even when there is no contention it would still debounce for 50ms. I think that simply reinserting the update at the end of the message queue should still handle things pretty well but we can see the benchmarks.

@msfstef
Copy link
Contributor Author

msfstef commented Jan 29, 2025

benchmark this

Copy link
Contributor

github-actions bot commented Jan 29, 2025

Benchmark results, triggered for a8a1a

  • concurrent shape creation completed

concurrent shape creation results

  • diverse shape fanout completed

diverse shape fanout results

  • unrelated shapes one client latency completed

unrelated shapes one client latency results

  • write fanout completed

write fanout results

  • many shapes one client latency completed

many shapes one client latency results

Copy link
Contributor

@robacourt robacourt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! I've just left one comment

@msfstef msfstef merged commit eccdf9f into main Jan 29, 2025
33 checks passed
@msfstef msfstef deleted the msfstef/contention-on-recovery branch January 29, 2025 15:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants