Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

import: accept more file paths #101

Open
clbarnes opened this issue Jan 28, 2022 · 1 comment
Open

import: accept more file paths #101

clbarnes opened this issue Jan 28, 2022 · 1 comment

Comments

@clbarnes
Copy link
Contributor

clbarnes commented Jan 28, 2022

Given a lot of microscope output has very long names, I think a deep TIFF stack would quickly hit shell limitations on command length. Two ways round this:

  1. Accept input from stdin
  2. Accept a directory and discover files within it (sorting them) - probably better handled by piping fd | sort into option 1, but might be convenient for simple cases

Here's a repo which takes Vec<OsString>, where those OsStrings represent a mixture of file paths, directory paths, and - for stdin, and produces Vec<PathBuf> to files. Recursion into directories is optional and configurable.

https://github.com/clbarnes/filelist

@clbarnes clbarnes changed the title import-tiff: accept more file paths import: accept more file paths Jan 28, 2022
@aschampion
Copy link
Owner

Good issue. One mitigation and otherwise complementary, useful feature would just be to have a block offset to import into a (potentially pre-existing) dataset, with manually specified total dimension. This would make it easier to distribute parallel imports and limit the number of files given to any one command.

Eventually I see import going away in its current form once ndgest succeeds n5gest, because image stacks will just be another chunknd-implementing backend which can be "imported" via convert, and it will be easy to specify arbitrary sources via stdin streaming or autodiscovery like you suggest.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants