Audio Corruption After Applying patchtransformers Script

[mervegrbz]

I followed the setup steps in the README and tested the model with the following input:

lyrics: [verse]
Staring at the sunset, colors paint the sky
Thoughts of you keep swirling, can't deny
I know I let you down, I made mistakes
But I'm here to mend the heart I didn't break

[chorus]
Every road you take, I'll be one step behind
Every dream you chase, I'm reaching for the light
You can't fight this feeling now
I won't back down
You know you can't deny it now
I won't back down

[verse]
They might say I'm foolish, chasing after you
But they don't feel this love the way we do
My heart beats only for you, can't you see?
I won't let you slip away from me
sequence: 3 
tokens: 3000

When running the model without the patchtransformers script, the output generation was extremely slow (1.5 minutes of audio took ~23 minutes to generate).

After applying the patchtransformers script, the performance improved (1.5 minutes of audio generated in ~10 minutes). But, the output audio was completely broken—only weird noises were produced instead of actual speech or singing.

Has anyone else encountered this issue? Are any specific debugging steps recommended to check where corruption occurs?

Thanks a lot

[deepbeepmeep]

the random seeds are changed because i use a different 'attention' agorithm so it is not possible to compare a song generation with and without the patch. I suggest you try different seeds. if one of them produces good stuff, this might confirm the seed was not good in the first place