Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle client resets during schema migrations + integration tests #7106

Conversation

danieltabacaru
Copy link
Collaborator

@danieltabacaru danieltabacaru commented Nov 2, 2023

What, How & Why?

During sync schema migrations, an uploaded change may trigger a client reset on the server. If the client reset is interrupted, then when the app restarts or the session is resumed the server sends only the client reset error (without sending the migration error with the previous schema version). The client therefore needs to store the previous schema version and have it available until the migration is completed. A new (private) metadata table schema_migration_metadata is added for this purpose.

In addition, integration tests have been added to test several scenarios.

Fixes #7023, #6956.

☑️ ToDos

  • [ ] 📝 Changelog update
  • 🚦 Tests (or not relevant)
  • [ ] C-API, if public C++ API changed.

@danieltabacaru
Copy link
Collaborator Author

danieltabacaru commented Nov 2, 2023

@michael-wb Let's review the tests and the other changes and not worry about tests failing for now. Once I merge this into the feature branch, I will rebase master (which is still pending some changes from James), and then the tests should pass.

@danieltabacaru danieltabacaru marked this pull request as ready for review November 2, 2023 12:36
Copy link
Contributor

@michael-wb michael-wb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good - a few questions and nits.

test/object-store/util/sync/baas_admin_api.cpp Outdated Show resolved Hide resolved
test/object-store/sync/flx_schema_migration.cpp Outdated Show resolved Hide resolved
Copy link
Contributor

@michael-wb michael-wb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@danieltabacaru danieltabacaru merged commit e94cd37 into feature/sync_schema_migrations Nov 9, 2023
2 checks passed
@danieltabacaru danieltabacaru deleted the dt/schema_migration_integration_tests branch November 9, 2023 08:52
danieltabacaru added a commit that referenced this pull request Nov 14, 2023
)

* Add option to use draft deployments when creating a schema for baas tests

* Track sync schema migration between sessions + minor refactoring

* Integration tests

* Use Transaction refs instead of shared pointers
danieltabacaru added a commit that referenced this pull request Dec 14, 2023
)

* Add option to use draft deployments when creating a schema for baas tests

* Track sync schema migration between sessions + minor refactoring

* Integration tests

* Use Transaction refs instead of shared pointers
danieltabacaru added a commit that referenced this pull request Jan 8, 2024
)

* Add option to use draft deployments when creating a schema for baas tests

* Track sync schema migration between sessions + minor refactoring

* Integration tests

* Use Transaction refs instead of shared pointers
danieltabacaru added a commit that referenced this pull request Jan 15, 2024
)

* Add option to use draft deployments when creating a schema for baas tests

* Track sync schema migration between sessions + minor refactoring

* Integration tests

* Use Transaction refs instead of shared pointers
danieltabacaru added a commit that referenced this pull request Jan 22, 2024
* Add schema version to flexible sync client BIND message (#6863)

* Send schema version as part of BIND messages

* Add pause_async to SyncSession (#6845)

* Add async shutdown of SyncSession

* SyncSession::shutdown returns a Future instead of passing callbacks around

* Rename shutdown() to pause_async(). Hide it from the public API.

* Remove unneeded header import

* Upload data and migrate schema (#6944)

* Handle client resets during schema migrations + integration tests (#7106)

* Add option to use draft deployments when creating a schema for baas tests

* Track sync schema migration between sessions + minor refactoring

* Integration tests

* Bump sync protocol version to 11

* Update baas commit for evergreen tests

* Allow resetting schema version to zero

* Improve changelog
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 21, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants