Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

remove a replicated file on close at consumer #68

Open
JaeseungYeom opened this issue Jan 19, 2024 · 0 comments
Open

remove a replicated file on close at consumer #68

JaeseungYeom opened this issue Jan 19, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@JaeseungYeom
Copy link
Contributor

As the most simple cache management, we need to remove the file used at consumer side at the earliest chance.
If it is certain that a file is used only by a single consumer at a time, we can safely remove it on closing the file that has been open in read-only mode. If there is a chance that a file can be read by multiple consumers, we cannot remove on closing.
In this case, we can either rely on a lock for shared reading, and only remove a file when there is no other consumer has placed a lock on the file. There might be another option by maintaining a reference counter and update it based on inotify
We can expose an environment variable to enable a specific behavior.

This is for the simplest workflow scenario where a unique pair of producer and consumer shares files exclusively, and each file is read only once for the most part.
For other scenarios, we would need more advanced cache management schemes.

@JaeseungYeom JaeseungYeom added the enhancement New feature or request label Jan 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant