Singularity image does not build

[clbarnes]

To reproduce:

cd singularity
make

...
+ conda install tensorflow-gpu==1.3
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.

PackagesNotFoundError: The following packages are not available from current channels:

  - tensorflow-gpu==1.3

Current channels:

  - https://repo.anaconda.com/pkgs/main/linux-64
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/r/linux-64
  - https://repo.anaconda.com/pkgs/r/noarch

[sheridana]

The singularity image used for paper experiments was built a while ago and the tensorflow version is now deprecated. I can share the built image if you want to use the current network implementations (it is just too large to upload to github). Alternatively, you could refactor to use pytorch and funlib.learn.torch which makes things a bit cleaner.

[clbarnes]

Bumping the version to 1.15 allows it to build, and if TF respects semver, nothing should have broken. Some of the tests don't pass, although that seems to be for unrelated reasons.

[sheridana]

I do recall that some things in TF changed between those versions (i.e small function name changes, deprecations etc), so you would just have to update the network scripts accordingly

[sheridana]

As for some of the tests failing, these are old and should be updated or removed. We haven't done anything with sqlite, luigi, hdf, or lsd based agglomeration in a while.

[clbarnes]

Great, thanks. Also thanks for uploading the tutorials, they'll be a big help. What's the address to reach you now, or is it best to talk to Jan about getting the images and trying to make use of this? I think I still have my Janelia address (barnesc).

[clbarnes]

Ah, I do have it but can't easily log in - try my LMB address cbarnes AT mrc-lmb.cam.ac.uk

[clbarnes]

Could malis be removed from these images? It has very restrictive build constraints and is not used by lsd directly; it's also not actually used in any of the downstream cases I've looked at yet where it's imported.

[sheridana]

Yeah my Janelia email ended, just sent you an email. Yep malis could be removed, I believe it was only needed for training the malis loss networks (which we compared against in the study). I will create a new singularity container for training with pytorch soon, the only unfortunate thing is that the pretrained paper networks all used the image from the current build, in which everything is very deprecated.