MLPF training with CLIC simulation

MLPF as in the upcoming paper, with training on CLIC simulation.
Cleanup of v1.5 with BFG to remove LFS errors.

What's Changed

• add license (Apache 2.0) by @jmduarte in #86
• hep_tfds support in raytune by @erwulff in #82
• update with larger dataset 1.2.0 [TF] by @jpata in #84
• Add raytune algorithms by @erwulff in #85
• Minor raytune fixes by @erwulff in #87
• Load best weights before saving model after end on training by @erwulff in #88
• Add Bayesian Optimization to raytune command, using nevergrad or scikit-optimize by @erwulff in #89
• ACAT2021 benchmark by @jpata in #92
• New best hyperparameter config by @erwulff in #90
• Rewrite tf.einsum using tf.math.multiply by @jmduarte in #93
• Update plots for ACAT'21 based on PPD suggestions by @jpata in #94
• Specify version of Ray Tune in GitHub tests by @erwulff in #99
• Gen/Sim training dataset by @jpata in #100
• Fix raytune imports by @erwulff in #106
• Better CMS dataset, fix f16, fix transformer by @jpata in #105
• Fix output decoding for PFNetDense by @jpata in #107
• feat: Ray Tune analysis on JUWELS by @erwulff in #111
• Up flatiron modules by @erwulff in #112
• fix optimizer save/load, add mpnn config, get f16 training to work by @jpata in #108
• updated/documented lrp and pytorch pipeline by @farakiko in #110
• Fix PCgrad loading, trainable weights by @jpata in #113
• Fix quickstart nb by @erwulff in #116
• Comet-ml offline logging by @erwulff in #115
• optimized pytorch geometric pipeline using DDP by @farakiko in #118
• Fix bug in CustomCallback class by @erwulff in #119
• Hypertuning development by @erwulff in #120
• Ray cleanup by @erwulff in #121
• June 2022 update: new datasets, jet/MET level validation, additional loss terms by @jpata in #114
• Add multinode training using Horovod by @MaPoKen in #104
• log jet/met reso, make event loss configurable, add sliced Wasserstein loss by @jpata in #123
• fix small bug in eval by @jpata in #127
• Gen jet loss by @jmduarte in #126
• Faster test, pre-commit formatting, general cleanup by @jpata in #129
• Pre commit fixes by @farakiko in #131
• Comparison job for different event losses by @jpata in #132
• Fix lr logging by @erwulff in #137
• integrate hep_tfds, September 2022 benchmark training by @jpata in #136
• MET loss as an option by @jpata in #138
• added MET file by @jpata in #139
• Fix MET loss, validation in CMSSW by @jpata in #141
• Bump tensorflow from 2.9 to 2.9.1 by @dependabot in #143
• Ray Tune checkpointing fix, allow LR schedules for non-PCGrad opt, and more. by @erwulff in #142
• PCGrad with LR schedules, resume from checkpoint with LR schedules by @erwulff in #145
• Add ability to train on Habana Gaudi by @jmduarte in #135
• high-pT gun samples by @jpata in #144
• Additional gun samples, move padding from dataset to model, change response plot definition, update transformer model by @jpata in #146
• Add benchmarking utilities by @erwulff in #147
• added clic pipeline from parquet by @jpata in #149
• added acat2022 model by @jpata in #148
• fix num_cpus flag by @jpata in #150
• Bump tensorflow from 2.10.0 to 2.10.1 by @dependabot in #151
• training on LUMI HPC by @jpata in #152
• Refactoring, CLIC datasets by @jpata in #153
• format black by @jpata in #154
• ssl-based mlpf first iteration by @farakiko in #158
• Fix legacy CLIC dataset pdgid by @jpata in #160
• edm4hep postprocessing by @jpata in #159
• SSL updates: pipeline, new datasets and jet clustering by @farakiko in #161
• tune the pytorch MLPF model to be more similar to TF by @jpata in #165
• Raytune updates, LR-finder bug fix by @erwulff in #164
• tuning the downstream MLPF model [pytorch] by @jpata in #166
• Ssl finetuning by @farakiko in #167
• Update data split mode for SSL studies [pytorch] by @farakiko in #168
• few fixes and cleanups to pytorch, update sim scripts by @jpata in #169
• update clic plots by @jpata in #171
• fix: error in raytune search space by @erwulff in #170
• optimizing VICReg by @farakiko in #173
• update CLIC dataset, retrain MLPF by @jpata in #172
• update README by @jpata in #175
• Refactoring by @farakiko in #174
• clean up dataset prep by @jpata in #176
• fix loader, readd tensorboard by @jpata in #177
• Additional small fixes to pytorch by @jpata in #178
• standardize input features, re-enable fp16 [TF], unify plotting [pytorch] by @jpata in #179
• Pin torch to 1.13.0 by @jpata in #180
• CLIC new samples with 1M events by @jpata in #181
• CLIC new datasets, hit based training option by @jpata in #182
• update hit-based training by @jpata in #184
• TF perf tuning, CLIC benchmarks, flatiron scripts by @erwulff in #185
• Add inference command by @erwulff in #187
• clean up repo by @jpata in #188
• scale test on lumi, fix horovod by @jpata in #189
• switch to tfds array_record, improve visualization, dataset descriptions by @jpata in #190

New Contributors

• @MaPoKen made their first contribution in #104

Full Changelog: v1.4...v1.5