Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reproducing the paper results #6

Open
LSunQQ opened this issue Mar 29, 2023 · 0 comments
Open

Reproducing the paper results #6

LSunQQ opened this issue Mar 29, 2023 · 0 comments

Comments

@LSunQQ
Copy link

LSunQQ commented Mar 29, 2023

Thank you for your exciting work and very clean code. I am having trouble reproducing the results mentioned in the paper and would appreciate it if you could help me.

  1. Reproducing the results of cifar100
    I was trying to get the scores on the samples from Table 2 in the paper. However, the results on all datasets did not match the ones that I see in the paper. I ran your code twice with "python3 train.py --dataset cifar100 --lbl-percent 10 --novel-percent 50 --arch resnet18". The result on seen,novel, all class is 1-2% lower than that in your paper.
    logcifar100_label10_1.txt
    logcifar100_label10_2.txt

  2. With 50% label,The performance on the CIFAR-100 dataset with 50% label,the paper suggested adjusting the temperature to 0.2. However,the results on novel class is 41% while 49% in paper.
    logcifar100_label50_t0.2.txt
    with temperature set to 0.1, the result on novel class is 1% lower than that in your paper .
    logcifar100_label50_t0.1.txt

  3. I noticed that the CosineAnnealingLR with warm-up mentioned in your paper has been removed.

Thanks a lot for your time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant