Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make pft.imsave replacebly by skimage.io imsave #6

Open
haesleinhuepf opened this issue Sep 8, 2022 · 3 comments
Open

Make pft.imsave replacebly by skimage.io imsave #6

haesleinhuepf opened this issue Sep 8, 2022 · 3 comments

Comments

@haesleinhuepf
Copy link
Member

A common usecase would be that someone writes a notebook on their laptop with

from skimage.io import imsave

Then, they want to run it on the cluster, where the code would be

pft = bt.ProjectFileTransfer(source_mount, target_project)
imsave = pft.imsave

It would be cool if this was simpler...

@haesleinhuepf
Copy link
Member Author

Actually, that's a cool idea, because also this would work:

from pandas import from_csv

and alternatively:

pft = bt.ProjectFileTransfer(source_mount, target_project)
from_csv = pft.pandas_from_csv

@haesleinhuepf
Copy link
Member Author

haesleinhuepf commented Sep 8, 2022

Alternatively, if this would work, we could switch such an environment:

bt.ProjectFileTransfer("C:/temp", "C:/Robert/Data")

... I think that's the most convenient way

@thawn
Copy link

thawn commented Oct 19, 2022

the function signatures of the various save and read functions are now the same as for the original functions.
i.e. I removed the target parameter from the save functions (and replaced it by an attribute of projectFileTransfer).
This means that now something like
imsave = pft.imsave should work (need to do more testing, though)

In principle, this should even work locally, because pft always looks if it finds the file locally before trying to download it from the fileserver. Also, pft saves the file locally, if it can write to the parent directory (except if the parent directory is the current working directory)

however, there are a few open tasks:

  • handle automatic conversion of windows to linux paths (use pathlib for that)
  • how to deal with saving files to the current working directory? We almost always have write access there, but what about if the user wants to save to the fileserver instead? (therefore, we currently save to the fileserver/project space instead of the current working directory) -> configure when initializing the pft object
  • how to deal with pandas.to_x methods? If we really wanted this to work as a plug-in replacement, we would have to implement our own pandas dataframe, that inherits from the original and overrides those methods. - a bit daunting (because pandas has so many of these methods and we would have to override them all) -> leave this as is for now, until it is needed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants