Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add first draft of indexing ADR. #34

Merged
merged 13 commits into from
Jul 18, 2024
Merged

Add first draft of indexing ADR. #34

merged 13 commits into from
Jul 18, 2024

Conversation

tpendragon
Copy link
Contributor

@tpendragon tpendragon commented Jul 9, 2024

Refs #30

@tpendragon tpendragon marked this pull request as ready for review July 9, 2024 20:49
Copy link
Member

@hackartisan hackartisan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is very clearly laid out, and I like the way the performance reasoning is in each section.

This descriptions in this document focus primarily on incremental changes (although I think the performance metrics are oriented around a full cache refresh in each case). We haven't talked as much about a full refresh of all 3 caches. Should we try to futher specify that process more here or address it separately?


| id | data | log_order |
|------|-------|-----------|
| UUID | JSONB | INT |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the same log order value as the previous table, or different?

@tpendragon
Copy link
Contributor Author

This descriptions in this document focus primarily on incremental changes

That's interesting, it was totally my intent that this ADR be mostly about a full cache refresh, with a note that incremental changes should be much faster. I'll have to re-read with this viewpoint in mind.

@hackartisan
Copy link
Member

This descriptions in this document focus primarily on incremental changes

That's interesting, it was totally my intent that this ADR be mostly about a full cache refresh, with a note that incremental changes should be much faster. I'll have to re-read with this viewpoint in mind.

I think it's that each description says "every minute..."

Also I think it would be good to talk about the life of the event log. if we do a full cache refresh, then another one, does the event log just get bigger? Do we know where to start to read the latest of everything?

@tpendragon
Copy link
Contributor Author

Oh good read, we need another column that's a consistent ID so we can compact it. So that if it exists already we UPDATE.

hackartisan and others added 12 commits July 17, 2024 13:58
Co-authored-by: Amin Zare <[email protected]>
Co-authored-by: Trey Pendragon <[email protected]>
Co-authored-by: Eliot Jordan <[email protected]>
Co-authored-by: Shaun Ellis <[email protected]>
Co-authored-by: Amin Zare <[email protected]>
Co-authored-by: Trey Pendragon <[email protected]>
Co-authored-by: Eliot Jordan <[email protected]>
Co-authored-by: Shaun Ellis <[email protected]>
Co-authored-by: Amin Zare <[email protected]>
Co-authored-by: Trey Pendragon <[email protected]>
Co-authored-by: Eliot Jordan <[email protected]>
Co-authored-by: Shaun Ellis <[email protected]>
Remove verification, add consequences, reference the book.

Co-authored-by: Anna Headley <[email protected]>
Co-authored-by: Amin Zare<[email protected]>
Co-authored-by: Eliot Jordan <[email protected]>
Co-authored-by: Shaun Ellis <[email protected]>
@hackartisan hackartisan merged commit 39fb0bf into main Jul 18, 2024
1 check passed
@hackartisan hackartisan deleted the 30-adr branch July 18, 2024 20:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants