Set a fixed seed for all steps

[cflerin]

Especially important for dimensionality reduction, clustering, etc.

[dweemx]

I think we should do the same for SCENIC.
We should save the seeds (or use the run ID as a seed) for each of the runs so that the results should be reproducible.

[cflerin]

Good idea. But there are some things internal to pySCENIC that would need this addition as well. One thing in particular is the AUCell step, when calculating the cell rankings, ties are broken dependent on the random seed (aertslab/pySCENIC#99). This is a fix that would need to be implemented on the pySCENIC side. I believe the same applies to the ranking in the cisTarget step as well. For GRNBoost2/Genie3, there is already an option to set the seed in the pySCENIC CLI, but this needs to be implemented in the NF process here also.

[dweemx]

https://medium.com/@ODSC/properly-setting-the-random-seed-in-ml-experiments-not-as-simple-as-you-might-imagine-219969c84752

[dweemx]

The user should be able to use a custom seed => put the seed as a param part of the config.

[dweemx]

Useful links

• score_genes reproducibility scverse/scanpy#313
• https://github.com/rstudio/reticulate/blob/master/R/seed.R

[dweemx]

Current stochastic processes:

• SC__SCANPY__NEIGHBORHOOD_GRAPH (scanpy repo)
• SC__SCANPY__DIM_REDUCTION (scanpy repo): concerns only UMAP
• PCACV__FIND_OPTIMAL_NPCS (pcacv repo)
• GRNBOOST2_WITHOUT_DASK (scenic repo)

[cflerin]

Nice work on this @dweemx ! But it's not quite finished for the pySCENIC steps. Maybe it would be better if we start another issue in that repo? It would need work on the pySCENIC codebase before we're able to implement anything here.

[dweemx]

What did I miss ?
I think I set the seed for the GRN step ;
Is there also some randomness in the cistarget/AUCell steps ?
Yeah, good idea then to create another issue in the scenic repo 👍

[cflerin]

No you didn't miss anything. But there some things internal to pySCENIC (ctx and aucell steps) that are still stochastic. I'll start an issue there.

[dweemx]

This regards the following processes:

• SC__SCANPY__DIM_REDUCTION

The seed is not passed to random_state parameters in:

• sc.tl.tsne
• sc.tl.umap
• sc.tl.pca

[dweemx]

Fully reproducible workflows from version v0.13.0

[erwinlagu]

Good Job