Dockerizing SRBench

[lacava]

To update you all, I have been working on refactoring the build process so that each algorithm has its own docker environment. This makes install much more configurable for each algorithm. By default algorithms use a shared dockerfile running micromamba, but contributors can choose to upload their own Dockerfile with their algorithms if they want.

I’ve been able to reduce the average build time to ~ 3 minutes, and also push the images to dockerhub so that you don’t have to build the images if you don’t want to. (After some work I got build sizes down to 1-2 GB for most algorithms, but there’s probably room for improvement there).

This is happening on the docker compose branch and the build is pretty much working except for 1 algorithm (https://github.com/cavalab/srbench/actions/runs/9984875946).

The basic build procedure is now to generate the docker compose file

bash scripts/make_docker_compose_file.sh

which generates a docker-compose.yml file with a service for each algorithm, and then run:

• docker compose pull to pull the images or
• docker compose build to build the images or
• docker compose run [algorithm] bash test.sh to run the tests for [algorithm].

you can send commands to be run in the same way, e.g.

docker compose run afp python evaluate_model.py ...

[alcides]

Can you provide an example of how to run the evaluation of a single algorithm?

I am running docker compose run geneticengine python evaluate_model.py ../pmlb/eq1/eq1.tsv -ml geneticengine -seed 42, but the docker image has no access to "../pmlb", which is where datasets are located.

[lacava]

tbh i have not tested this functionality yet, only run the tests. The issue there is docker compose only mounts the experiments folder as a volume. we either need to add another volume, move pmlb and/or use its cache / api.

[lacava]

i can't test now but i would try something like

- ./pmlb:/pmlb

to the volumes in

srbench/scripts/make_docker_compose_file.sh

Line 40 in 8a78e47

- ./experiment:/srbench

and running make_docker_compose_file.sh again, then running that command.

[alcides]

I expected as much (there is so much to test when moving to containers).

I think either we need docker-compose to also export the datasets (which is different on each machine - problematic!).

The alternative is to use the pmlb python interface (and maybe generate the csv temporarily), but that limits people adding their own dataset without adding python code.

I am willing to help here, but I would need to clarify the direction to take first.

[alcides]

Btw, your suggestion worked perfectly!