Feature request: Add an option to the Polyak SGD optimizer that uses only the positive part of the suboptimality gap. That is, it replaces $f(x) - f^*$ with $(f(x) - f^*)_+$, where $a_+ = \max \{a, 0\}$ is the ramp function.

This is the variant of Stochastic Polyak Stepsize (SPS) that Garrigos et al. 2023 call SPS$_+$.

Possible names for this option are

• clip
• clamp
• plus

I can submit a PR for this.