Split long project tests into different environments such that they run concurrently

[brownbaerchen]

I got really tired of waiting so long. I just added environments for the longest running tests. Let me know if you want more or less granular. Also, the environments are the same as the base environment, since only these tests took so long. Tests needing other environments can stay there as far as I am concerned.

[lisawim]

Hey, this is really cool! Now the tests run much faster! ❤️

[tlunet]

Love it !

[brownbaerchen]

Feel free to suggest a better solution regarding the environment files. I just did the easiest thing I could think of, but it's kind of nasty.

[pancetta]

Nice! But also nasty.. I'd appreciate if we could come to a version where each project ships with it's own env file. Then we'd need a loop over the projects and "all" is "parallel". The explicit list of environments everywhere is somewhat ugly and error-prone. I guess we'd have to move the tests to the project files and require a default name for the environment file? Would that work? @jakob-fritz what do you think?

[jakob-fritz]

@pancetta Yes, the layout, that you proposed is possible (double checked with pytest-docs ). Then, the tests would go in a sub-dir of each project (pySDC/projects/NiceProject/tests/test_awesome.py). The environment-file could go into each project as well. The default name is environment.yml, so we don't need to come up with a default name.
However, this also means duplication of files with identical content (which is always a not-so-nice approach). An idea could be using the environment-base.yml as it is now and using symlinks in the project-dirs where no special environments are needed. As i read it online, you can check in symlinks are you can do with "regular" files. So this might work and reduce duplication increasing maintainability.

[pancetta]

Ok, but duplication only refers to the environment.yml, yes? So, we would add sym-links to the projects which don't require any special environment file (besides the base one, maybe also besides other default ones like MPI or so).

The projects would need to be listed in the CI file, right?

[jakob-fritz]

Yes, correct. The projects would still (all) need to be listed explicitly in the CI-file. So the symlinks would only get rid of the duplication of environment.yml-files.

[pancetta]

Any volunteers? Otherwise this will have to wait for some long night of mine..

[jakob-fritz]

I can do it. But this means, that the fixing of cupy (see #424 )will have to wait until this is done. Nevertheless, I would prefer this new order (first this, then the MPI-NCCL issue) as then, the CI runs faster, which leads to better/easier/faster development in general.

[pancetta]

Thoughts, @brownbaerchen ?

[brownbaerchen]

Sounds good to me!

[pancetta]

Well, then, fire at will, @jakob-fritz. The goal is to make the projects as independently as possible, without too much duplication. Good luck!