Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add DB subscription #114

Merged
merged 1 commit into from
Aug 26, 2024
Merged

Add DB subscription #114

merged 1 commit into from
Aug 26, 2024

Conversation

richardhuaaa
Copy link
Contributor

@richardhuaaa richardhuaaa commented Aug 21, 2024

Adds a struct that can subscribe to changes in a database query, starting from some cursor onwards. Features:

  • Poll based on an interval or on an input notifier channel
  • Serve updates via an output channel
  • Stops polling when context is closed
  • Non-racy unit tests (hopefully)

Also adds a flexible DB query against the gateway_envelopes table that can query based on a range of params. Given the size of the query, I should add a unit test for it eventually.

Copy link
Contributor Author

richardhuaaa commented Aug 21, 2024

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @richardhuaaa and the rest of your teammates on Graphite Graphite

@richardhuaaa richardhuaaa force-pushed the 08-20-add_queries_and_sprocs branch 3 times, most recently from 966eb8b to b342aa5 Compare August 21, 2024 04:30
@richardhuaaa richardhuaaa force-pushed the 08-20-add_db_subscription branch from 647b702 to 92f5ac8 Compare August 21, 2024 04:36
pkg/db/subscription.go Outdated Show resolved Hide resolved
Base automatically changed from 08-20-add_queries_and_sprocs to main August 21, 2024 16:38
@richardhuaaa richardhuaaa force-pushed the 08-20-add_db_subscription branch 3 times, most recently from a100969 to 4467348 Compare August 23, 2024 07:38
@richardhuaaa richardhuaaa force-pushed the 08-20-add_db_subscription branch from 4467348 to db44f65 Compare August 26, 2024 05:36
@richardhuaaa richardhuaaa marked this pull request as ready for review August 26, 2024 05:39
OR topic = @topic)
AND (sqlc.narg('originator_node_id')::INT IS NULL
OR originator_node_id = @originator_node_id)
AND (sqlc.narg('originator_sequence_id')::BIGINT IS NULL
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unrelated to the PR, but who generates the sequence ID?

Copy link
Contributor Author

@richardhuaaa richardhuaaa Aug 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great question - the originator does, this would be extracted from the originator_sid field of UnsignedOriginatorEnvelope proto.

As for how the originator allocates these ID's in the first place, it is created by the BIGSERIAL id field of the staged_originator_envelope table - so we're using a postgres sequence for this.

@richardhuaaa richardhuaaa merged commit 9771fb2 into main Aug 26, 2024
5 checks passed
@richardhuaaa richardhuaaa deleted the 08-20-add_db_subscription branch August 26, 2024 22:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants