Command-line capability of running cross-validation

[dahhei]

Feature request for being able to run cross-validation from the command line. When a project has many videos for cross-val, the all k-fold cross-val takes a long time. 😕

[SkepticRaven]

It's not documented, but it looks like https://github.com/KumarLabJax/JABS-behavior-classifier/blob/main/stats.py does exactly this, when you pass a -k greater than the number of possible k-folds. It only operates on exported classifiers in its current state.

There's a handful of upgrades to it that would make it more useful:

• Allow either project folders or saved classifiers.
• Add an easier way to just say "do all k-fold".
• Allow easy access to window sizes and feature subsets/setup routines for automated hyperparameter optimization
  • Requires projects, since features would need to be re-calculated
  • (Longer term) Add in Nicks efforts for ML hyperparameters (f664540)