Review Tutorial 13 on SBC based diagnostic methods (#1051)

* fix: import statement for sbc_rank_plot * fix: explanation of c2st in this tutorial * fix: torch UserWarning when calculating std on one value * add: demonstration of nD SBC mapped to 1D finished removed metadata from notebook * reformatted tutorial * fix: formatting of sbc module * fix: formatting issue * fix: trimmed down multi-dimensional SBC discussion * fix: copy and paste error denominating the wrong number of dimensions * fix: reworded hard-to-understand explanation of reduce_fns * fix: avoid confusion about single dimension Co-authored-by: Jan <[email protected]> --------- Co-authored-by: Jan <[email protected]>
sbi-dev · Mar 25, 2024 · c383d7f · c383d7f
1 parent 32e365d
commit c383d7f
Show file tree

Hide file tree

Showing 2 changed files with 670 additions and 177 deletions.
diff --git a/sbi/diagnostics/sbc.py b/sbi/diagnostics/sbc.py
@@ -361,7 +361,8 @@ def check_uniformity_c2st(
     ])
 
     # Use variance over repetitions to estimate robustness of c2st.
-    if (c2st_scores.std(0) > 0.05).any():
+    c2st_std = c2st_scores.std(0, correction=0 if num_repetitions == 1 else 1)
+    if (c2st_std > 0.05).any():
         warnings.warn(
             f"""C2ST score variability is larger than {0.05}: std={c2st_scores.std(0)},
             result may be unreliable. Consider increasing the number of samples.