Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Evaluate SOP using pretrained model, R@1 much lower from the number on README.md #27

Open
junwei-h opened this issue Jan 25, 2023 · 0 comments

Comments

@junwei-h
Copy link

junwei-h commented Jan 25, 2023

Hello,

I guess the README reported training result, not evaluation for SOP? I used your pre-trained model, ran the evaluate.py and got much lower result. Is this expected?

Method Backbone R@1 R@10 R@100 R@1000
Proxy-Anchor512 Inception-BN 79.2 90.7 96.2 98.6
Run code/evaluate.py Inception-BN 49.4 65.0 78.8 91.3

Here is what I did
python Proxy-Anchor-CVPR2020/code/evaluate.py --gpu-id -1 --batch-size 120 --model bn_inception --embedding-size 512 --dataset SOP --resume ../pretrained/SOP_bn_inception_best.pth --workers 4

To get it run on CPU, Ubuntu 20.04 (WSL), torch==1.13.1, I changed code related to cuda:

# model = model.cuda() 
if args.gpu_id != -1:
    model = model.cuda()

torch.load(args.resume, map_location=torch.device('cpu'))

Also fixed an error by adding strict=False

model.load_state_dict(checkpoint['model_state_dict'], strict=False)

Otherwise, I will hit errors raised by torch shown below

        if strict:
            if len(unexpected_keys) > 0:
                error_msgs.insert(
                    0, 'Unexpected key(s) in state_dict: {}. '.format(
                        ', '.join('"{}"'.format(k) for k in unexpected_keys)))
            if len(missing_keys) > 0:
                error_msgs.insert(
                    0, 'Missing key(s) in state_dict: {}. '.format(
                        ', '.join('"{}"'.format(k) for k in missing_keys)))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant