Skip to content

Commit

Permalink
added a race condition comment
Browse files Browse the repository at this point in the history
  • Loading branch information
icehaunter committed Jan 29, 2025
1 parent 4734090 commit 4a5aec3
Showing 1 changed file with 17 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,23 @@ defmodule Electric.ShapeCache.FileStorage.Compaction do
alias Electric.ShapeCache.FileStorage.KeyIndex
alias Electric.ShapeCache.FileStorage.ActionFile

# Compaction and race conditions
#
# `FileStorage` has a pointer to the last compacted offset (and the compacted log file name)
# which is updated atomically once the compaction is complete, so while it's ongoing, the
# pointer is not updated.
#
# While the log is compacted in place, it's actually a merged copy that's being
# compacted, not the original log. Original log is deleted after the compaction
# is complete and the pointer is updated.
#
# Any concurrent reads of the log that's being replaced are also OK: the `File.rename`
# on linux doesn't close the original file descriptor, so the reader will still see
# the original file, and we don't reuse file names. Any readers mid-file of the
# log that's being replaced but that read the chunk will continue from a correct chunk
# of the new file due to offset ordering being preserved. They might observe some updates
# more than once in a compacted form.

@spec compact_in_place({String.t(), String.t(), String.t()}, non_neg_integer(), (any(), any() ->
any())) ::
{String.t(), String.t(), String.t()}
Expand Down

0 comments on commit 4a5aec3

Please sign in to comment.