Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sync: Add transaction support #80

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from

Conversation

greezybacon
Copy link
Collaborator

@greezybacon greezybacon commented Oct 22, 2018

In this mode, the files synced in both the pull and push phases are not put into place until the completion of the phase. Instead, the updated files are staged into a .bitpocket/staging folder. Once all the changes are staged, they are moved into place. If the phase is interrupted, perhaps due to network connection quality, then no changes are reflected. This is especially useful when the changes being synchronized include software or data whose changes should be reflected as a whole.

This has the caveat of requiring a bit of extra space on the source and destination systems as extra space for all changes is required. However, it has the advantage that, if interrupted, the staged partial files can be used when resuming the transfer. This can greatly speed syncs over shaky network connections.

In this mode, the files synced in both the pull and push phases are not put
into place until the completion of the phase. Instead, the updated files are
staged into a `.bitpocket/staging` folder. Once all the changes are staged,
they are moved into place. If the phase is interrupted, perhaps due to network
connection quality, then no changes are reflected. This is especially useful
when the changes being synchronized include software or data whose changes
should be reflected as a whole.

This has the caveat of requiring a bit of extra space on the source and
destination systems as extra space for all changes is required. However, it has
the caveat that, if interrupted, the staged partial file can be used when
resuming the transfer. This can greatly speed syncs over shaky network
connections.
@greezybacon
Copy link
Collaborator Author

greezybacon commented Oct 22, 2018

@raphyduck, I included the spirit of your changes; however, in my experimentation, using --partial-dir results in clobbering of files and errors, if for instance you have files with the same name in multiple folders. However, using the --temp-dir option instead with --partial-dir implements the feature with complex folder structures without the clobbering. I also added --delete-delay which will delay deletes until the end of the sync process when the temp files are moved into place. I'd love to have your comments on the changes.


## Perform the transfer as a transaction. If interrupted, no updates are made.
# Partial file transfers remain to speed up future syncs. This should be
# combined with the "clean" command to automatically clean up ab
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oops. Is there going to be a "clean" command?

@raphyduck
Copy link

Very nice @greezybacon

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants