Unable to reproduce results for NYT-Fine

[divyagupta25]

Hi,

Thanks for sharing your work! I tried to run the given script for the NYT-Fine dataset but getting the below error -

The first iteration runs fine, and this happens during the second iteration. After head token fine tuning is completed, inference is done on the dataset and the top predictions are obtained from the get_pseudo_label function. From the total list, It looks like there are no pseudo labels for the classes business and abortion. Could you please help?

[yzhan238]

It may happen for rare classes to have very few pseudo labels. I didn't encounter this no-label problem when I run with the parameters in the provided script (scripts/NYT-Fine.sh). You can try to run it again. If the problem persists, you may adjust the threshold thres_p to let more pseudo labels in and see if it helps.

[divyagupta25]

Thanks for the information. Could you please share if you used any seed, as I am still unable to reproduce the results?