Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compactions: retain _first_ not last. Clever idea by @kostas #47

Open
markpapadakis opened this issue Nov 17, 2016 · 1 comment
Open
Assignees

Comments

@markpapadakis
Copy link
Member

@kostas on @dist-sys(Slack) came up with a great idea. Instead of retaining the last message in a set of messages with the same key, retain the first instead (during compactions). That way, you can implement an indempotency scheme, where you really care for the first message, and if you get more you will ignore them anyway, and you want them to be purged, eventually.

@markpapadakis
Copy link
Member Author

We could also use another mode where we retain all messages (i.e not the first or the last message for a given key), which could be handy if you only want to merge multiple immutable segments into fewer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant