Fix FIFO group locking behaviour #294
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Previously a FIFO queue was only able to return 1 message per request as it locked the group right after encountering the first message. Group unlocking had a similar problem where any deleted message in a group would unlock the whole group, even if there were pending messages in that group.
Thomasvdam
force-pushed
the
fix/fifo-group-locking
branch
from
7e36d97 to
5f2f8dd
Compare
|
@Thomasvdam Thanks for taking a look at this. Overall the goal of the change looks good to me, but we're in the middle of a large scale feature. I can't let this in since it'll cause more deployment headaches than we can bare there right now. I'm sorry. Once we're finished with the JSON work, let's regroup on this and see how this fits in with whatever comes out of that. Thanks again for supporting us here - I really appreciate it. |
|
No problem at all! I’ll keep an eye out for the JSON update 😃 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
First of all thank you for this amazing software, it's already been incredibly helpful! While using it I noticed that my FIFO batches were always only a single message and after some investigation I noticed it was due to a small bug in the implementation.
I added a (perhaps too verbose) test to verify that my suspicion was correct and that the fix actually worked but that uncovered a bunch of other bugs in the FIFO group locking behaviour so the PR became a little bigger than intended.
This is my first serious Golang contribution so my apologies if it's not idiomatic or performant Go.
Edit): I found a small bug where message attributes were not set when receiving a batch of messages. I hope you don't mind that I added it to this PR, but if that's a problem I can also make a separate PR for just those changes.
Explanation of changes
Previously a FIFO queue was only able to return 1 message per request as it locked the group right after encountering the first message. Group unlocking had a similar problem where any deleted message in a group would unlock the whole group, even if there were pending messages in that group.
In addition messages could receive a receipt/timeout despite not being served if they were part of a locked group. By moving the receipt assignment to come after the group lock check this should no longer occur.
SendMessageBatchdid not extract message attributes. I added an expectation to an existing test to verify it indeed didn't work and updated the implementation to extract message attributes for allsendEntries.Related issues
I think this should close #212