Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

concurrent datalad operations? #286

Open
mgxd opened this issue Dec 5, 2018 · 2 comments
Open

concurrent datalad operations? #286

mgxd opened this issue Dec 5, 2018 · 2 comments

Comments

@mgxd
Copy link
Member

mgxd commented Dec 5, 2018

When concurrently converting across multiple participants while using --datalad, heudiconv terminates before completion due to git index.lock files and the dataset is not put under version control.

Perhaps OpenNeuroOrg/datalad-service can be some inspiration

@yarikoptic
Copy link
Member

yarikoptic commented Dec 5, 2018

Yeap, a known limitation of the git and even you greater degree of datalad. No easy way around unfortunately ATM.
OpenNeuro datalad backend unlikely to have this issue since unlikely had multiple datalad invocations done on parallel in the same dataset, and they don't manage a hierarchy of the datasets.
They only workaround is to not use --datalad mode ATM and perform datalad add later for the entire result. This would though cause some files lacking their metadata tags

@satra
Copy link
Member

satra commented Dec 6, 2018

one possibility may be to stage the info into separate uuid branches and then merge it later in one go after conversion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants