Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pglite-sync saves incorrect offset for shapes returning more than one chunk #509

Open
AntonOfTheWoods opened this issue Jan 23, 2025 · 2 comments

Comments

@AntonOfTheWoods
Copy link
Contributor

(+ some discussion on discord).

According to @thruflo , "The electric-offset header "Must be used as the value of the offset parameter in your next request."
https://electric-sql.com/openapi.html#/paths/~1v1~1shape/get". pglite-sync currently uses messageAggregator[messageAggregator.length - 1].offset

messageAggregator[messageAggregator.length - 1].offset,

As all operation-level offsets appear to be the same, regardless of what is returned by the electric-offset header, that means pglite-sync will save the incorrect offset for any shape that returns more than one chunk initially.

What appears needs doing is to keep the last valid Offset returned by stream.lastOffset (so ignore 0_inf), which you can get at the moment you receive the last chunk.

@samwillis
Copy link
Collaborator

Thanks for finding this @AntonOfTheWoods!

@thruflo
Copy link
Contributor

thruflo commented Jan 23, 2025

What's the right behaviour here -- is the last message offset different from the electric-offset value? If so, is that correct?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants