Report error when using fine-tuning-gemma script

[THULiusj]

I didn't modify the script. When use 4bit lora training, when starting training, it reports error of " masked_scatter_: expected self and source to have same dtypes but got BFloat16 and Float". How to resolve this error?

[merveenoyan]

interesting, everything seems to be on bf16, I will check. thanks for reporting!