Performance & parallelization

Batch

Warning Being able to run over one epoch with a certain batch size is not enough, as we have a broad distribution of graph sizes:

Histograms of sizes


Size in MB of single graph on disk.

Luckily, the distribution gets narrower if we sum the memory requirements of multiple graphs (here: 25):

So for a batch size of ~20, having 30% room should be sufficient.

• No truth cuts & before we started skipping OOM experiments: Maximum supported batch size on della with an 80GB GPU seems to be: 20 (definite failures observed for >= 21).
• For EC with pt > 0.4 and batch size 30, we already get 70% GPU eff.
• For EC with pt > 0.4 and batch size 50, we already get 99% GPU eff and use ~56GB GPU memory

Data loader Parallelization

Simply use 12 cores, because we are limited by GPUs anyway.