-
Notifications
You must be signed in to change notification settings - Fork 25
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
upload: Mention that expired tasks might be treated as unrecognized #641
Conversation
Question for reviewers: if we take this change for upload and aggregation, should we also do the same for collection? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The Aggregators need to prevent themselves from aggregating such reports indefinitely.
I think the Aggregators will generally evict "old" state (for some implementation-defined definition of "old"), which will effectively avoid the risk of needing to receive such reports indefinitely since reports touching on evicted state MUST be dropped with report_dropped
. However, evicting old state is not required by DAP, so this change SGTM.
Question for reviewers: if we take this change for upload and aggregation, should we also do the same for collection?
I don't see why not. We might want to add an additional note reminding implementors that the leeway is even more important in this case, since collection typically must occur after the batch interval, so without appropriate leeway the final batch might be uncollectable.
I'm fine with this in the upload interaction, but not in the aggregation or collection interactions. Task expiry is about limiting which client reports get aggregated into a task, but it seems reasonable to allow aggregation and collection to occur after aggregators stop accepting reports. If we forbid aggregation or collection as aggressively as this change suggests, then we run the risk of orphaning valid, aggregatable reports. I would think that garbage collection accounts for this on its own: an aggregator will throw away reports if the task they belong to is expired for too long, at which point the aggregation or collection fails as described in {#input-share-validation}. I'd support adding an implementor's note discussing this, or something in operational considerations. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
GitHub requires that I put something here if I choose "Request changes" but see my comments above
0e9b10f
to
f7cfe33
Compare
7493778
to
9507a16
Compare
2025/1/15 call: don't abort during aggregation or collection: (1) in practice there is backlog to process for expired tasks; (2) we already allow dropping reports when the releavant replay state is evicted. Let's keep this for upload, so the Leader has a tool to deal with malicious Clients, but disallow for aggreagtion/collection. |
f7cfe33
to
c59944f
Compare
9507a16
to
7b64103
Compare
7b64103
to
12d1271
Compare
d665b67
to
6853dfe
Compare
Clients are free to pick any timestamp they wish, even one in the validity range of a task that has expired. The Aggregators need to prevent themselves from aggregating such reports indefinitely. Add some text to the upload flow indicating that indicating an unrecognized task may suggest the client is either misconfigured or misbehaving.
12d1271
to
a94aff3
Compare
Stacked on #639.
Clients are free to pick any timestamp they wish, even one in the
validity range of a task that has expired. The Aggregators need to
prevent themselves from aggregating such reports indefinitely.
Add some text to the upload flow indicating that indicating an
unrecognized task may suggest the client is either misconfigured or
misbehaving.