new Prodigy optimizer version: signigicantly less VRAM

[dxqbYD]

Hey,

just a heads-up, there is a new version of the Prodigy optimizer that requires significantly less VRAM.

• new hyperparameter slice_p==11:
  saves 45% of VRAM overhead. There should be no downside. VRAM requirement on the level of AdamW
• set beta1 to 0:
  saves another 25% - this can have a quality impact, similar to Adafactor compared to AdamW

Details:
konstmish/prodigy#22
konstmish/prodigy#23
konstmish/prodigy#32

OneTrainer PR:
Nerogar/OneTrainer#550

[xos84292-hub]

I use kohya_ss GUI. Do I need to manually update the Prodigy optimizer version? What file should I replace? Just update python?
-----update----
I manually replaced the two 1.1.2 py files and the problem has been solved. Thank you for sharing.

[dxqbYD]

I use kohya_ss GUI. Do I need to manually update the Prodigy optimizer version? What file should I replace? Just update python? -----update---- I manually replaced the two 1.1.2 py files and the problem has been solved. Thank you for sharing.

only upgrading the version doesn't activate the new features. the parameters have to be set.

this issue is a heads-up for kohya or any other developer who would like to add support to kohya scripts. I cannot support individuals to make it work, unless you'd be willing to implement it properly and provide a PR to kohya.

[Koronos]

I opened a PR with this change, @dxqbYD that's true with the gui's that add specific windgets for each optimizer (like onetrainer) but Kohya allows set your own params in the box under the dropdown: