What value is reported for k-Fold Validation?

[filipsPL]

Hi,

I perform k-fold cross-validation in mode="Optuna". My question is what metric value (eg AUC) is reported in the outputs (README.md or leaderboard.csv)? I guess this is the average from all folds, but this is not stated explicitly anywhere.

If so, reporting standard deviation would be also useful?

best,
Filip

[pplonski]

Hi @filipsPL,

The metrics are computed from all samples in the case of cross-validation. The AutoML gathers predictions from all folds and then computes the metric.

[filipsPL]

Thanks for a quick reply.

Just to make sure: the algoritm is as follows (eg for 3-fold CV and AUC):

1. For each fold the predictions are made:

fold1 -> predictions1
fold2 -> predictions2
fold3 -> predictions3

2. all predicted values (predictions1 + predictions2 + predictions3) are combined
3. AUC is calculated

?

[pplonski]

Yes

[filipsPL]

Thanks!