Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add PostgreSQL database support #1238

Draft
wants to merge 20 commits into
base: main
Choose a base branch
from
Draft

Add PostgreSQL database support #1238

wants to merge 20 commits into from

Conversation

partim
Copy link
Member

@partim partim commented Nov 19, 2024

This PR adds support for using the PostgreSQL database as a storage backend for Krill in preparation for supporting fail-over scenarios for higher availability.

The PR integrates the storage abstraction previously implemented via the kvx crate back into Krill to allow a tight coupling of storage primitives with Krill’s requirements.

We have opted to not move the storage code to async functions as proposed in #1152. We think that storage should generally be fast enough that the gain from using async isn’t all that big. Using a thread pool and sync functions makes it easier to reason about what’s going on and should keep the code simpler.

@partim partim marked this pull request as draft November 19, 2024 12:42
@partim partim marked this pull request as ready for review December 5, 2024 11:50
@partim partim marked this pull request as draft January 30, 2025 15:10
@partim
Copy link
Member Author

partim commented Jan 30, 2025

I moved the PR back to draft status as I intend to make some changes to how the database backend works. Currently it uses transactions and restarts them if they fail which may result in unexpected data changes if the restarted transaction doesn’t re-read the current status. The file system backend uses locks to prevent concurrent access altogether. This can also be achieved in PostgreSQL with row level locks, so I want to switch to that strategy to have the same semantics in both backends.

@partim
Copy link
Member Author

partim commented Feb 7, 2025

I’ve now changed how the database deals with concurrent access to mimic how the file system backend does things: There is a separate table for locks and whenever Store::execute is called, the scope is written to the locks table with an exclusive row lock so that other accesses to that row are blocked.

Now we need a bunch of testing …

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant