Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't evaluate KeyError: 'query_f'` #26

Open
zubairahmed-ai opened this issue Jan 8, 2020 · 35 comments
Open

Can't evaluate KeyError: 'query_f'` #26

zubairahmed-ai opened this issue Jan 8, 2020 · 35 comments

Comments

@zubairahmed-ai
Copy link

I am running the python evaluate_st.py --name market1501 but I receive the following error

Traceback (most recent call last): File "evaluate_st.py", line 149, in <module> query_feature = result['query_f'] KeyError: 'query_f'

when I print the result I get this

`{'header': b'MATLAB 5.0 MAT-file Platform: posix, Created on: Thu Jun 6 22:53:34 2019', 'version': '1.0', 'globals': [], 'distribution': array([[[0. , 0. , 0. , ..., 0. ,
0. , 0. ],
[0.00527704, 0.00263852, 0.00791557, ..., 0. ,
0. , 0. ],
[0.06461538, 0.09230769, 0.07384615, ..., 0. ,
0. , 0. ],
...,
[0.00226757, 0.00453515, 0.00680272, ..., 0. ,
0. , 0. ],
[0. , 0. , 0. , ..., 0. ,
0. , 0. ],
[0. , 0. , 0. , ..., 0. ,
0. , 0. ]],

   [[0.06578947, 0.02631579, 0.02631579, ..., 0.        ,
     0.        , 0.        ],
    [0.        , 0.        , 0.        , ..., 0.        ,
     0.        , 0.        ],
    [0.08928571, 0.04464286, 0.07142857, ..., 0.        ,
     0.        , 0.        ],
    ...,
    [0.        , 0.00591716, 0.        , ..., 0.        ,
     0.        , 0.        ],
    [0.        , 0.        , 0.        , ..., 0.        ,
     0.        , 0.        ],
    [0.        , 0.        , 0.        , ..., 0.        ,
     0.        , 0.        ]],

   [[0.075     , 0.08214286, 0.07857143, ..., 0.        ,
     0.        , 0.        ],
    [0.02518892, 0.05793451, 0.04282116, ..., 0.        ,
     0.        , 0.        ],
    [0.        , 0.        , 0.        , ..., 0.        ,
     0.        , 0.        ],
    ...,
    [0.00208768, 0.        , 0.00417537, ..., 0.        ,
     0.        , 0.        ],
    [0.        , 0.        , 0.        , ..., 0.        ,
     0.        , 0.        ],
    [0.        , 0.        , 0.        , ..., 0.        ,
     0.        , 0.        ]],`

   ...,

   [[0.11363636, 0.04545455, 0.11363636, ..., 0.        ,
     0.        , 0.        ],
    [0.        , 0.04761905, 0.07936508, ..., 0.        ,
     0.        , 0.        ],
    [0.02777778, 0.01388889, 0.        , ..., 0.        ,
     0.        , 0.        ],
    ...,
    [0.        , 0.        , 0.        , ..., 0.        ,
     0.        , 0.        ],
    [0.        , 0.        , 0.        , ..., 0.        ,
     0.        , 0.        ],
    [0.        , 0.        , 0.        , ..., 0.        ,
     0.        , 0.        ]],

   [[0.        , 0.        , 0.        , ..., 0.        ,
     0.        , 0.        ],
    [0.        , 0.        , 0.        , ..., 0.        ,
     0.        , 0.        ],
    [0.        , 0.        , 0.        , ..., 0.        ,
     0.        , 0.        ],
    ...,
    [0.        , 0.        , 0.        , ..., 0.        ,
     0.        , 0.        ],
    [0.        , 0.        , 0.        , ..., 0.        ,
     0.        , 0.        ],
    [0.        , 0.        , 0.        , ..., 0.        ,
     0.        , 0.        ]],

   [[0.        , 0.        , 0.        , ..., 0.        ,
     0.        , 0.        ],
    [0.        , 0.        , 0.        , ..., 0.        ,
     0.        , 0.        ],
    [0.        , 0.        , 0.        , ..., 0.        ,
     0.        , 0.        ],
    ...,
    [0.        , 0.        , 0.        , ..., 0.        ,
     0.        , 0.        ],
    [0.        , 0.        , 0.        , ..., 0.        ,
     0.        , 0.        ],
    [0.        , 0.        , 0.        , ..., 0.        ,
     0.        , 0.        ]]])}`
@Wanggcong
Copy link
Owner

Do you run this code to get a st model?

python3 gen_st_model_market.py --name ft_ResNet50_pcb_market_e --data_dir "/home/huangpg/st-reid/dataset/market_rename/"

@zubairahmed-ai
Copy link
Author

No I used the pretrained model pytorch_result.mat

@zubairahmed-ai
Copy link
Author

If you could please answer this it will help me run it

@Wanggcong
Copy link
Owner

All of the steps are shown in readme.md. If you do not want to re-train a model. I suggest you carefully follow these steps:

  1. create a folder (suppose you name it A)
  2. pull the two pretrained models into that folder. (i.e., pytorch_result2.mat for the st model and net_last.pth for net weights) .
  3. extract features, test (appearance feature extraction)
    python3 test_st_market.py --PCB --gpu_ids 2 --name A --test_dir "/home/huangpg/st-reid/dataset/market_rename/"
    note: "/home/huangpg/st-reid/dataset/market_rename/" is your data path.
  4. evaluate (joint metric, you can use your own visual feature or spatial-temporal streams)
    python3 evaluate_st.py --name A

@zubairahmed-ai
Copy link
Author

Ok let me try them now as I dont want to re-train a model

@zubairahmed-ai
Copy link
Author

For anyone running this on Windows replace line 147 in test_st_market.py with this filename = path.split('\\')[-1] and line 153 with this if label[0:2]=='-1' or label[0:2]=='-1_c': and save yourself some headache/time

@zubairahmed-ai
Copy link
Author

@Wanggcong I am running out of memory on running step 3 above, I am not sure what this will output. If its going to create a model that I can use in step 4 then can you please provide that model for me to evaluate? I am just using this for eval at the moment so really need some quick help.

@Wanggcong
Copy link
Owner

You could use small batchsize for test if it occurs " out of memory". For example,

python3 test_st_market.py --PCB --gpu_ids 2 --name A --test_dir "/home/huangpg/st-reid/dataset/market_rename/" --batchsize 1

@Wanggcong
Copy link
Owner

Step 3 is not a model, it is used for feature extraction for images.

@zubairahmed-ai
Copy link
Author

Nope still not working with batch sizes of 4,2 and 1
Can you please share a pretrained model? Also how can I use that in a video?

@Wanggcong
Copy link
Owner

Pretrained models are already released in readme.md. (both google drive or baidu). No other pretrained models! And the steps are clear. I think some errors are not about my code.

@Wanggcong
Copy link
Owner

If you want to apply this code to your scenario, I think you could re-train the st (spatial-temporal) model. Because different scenarios have different st relationships. Directly using this code for your scenario could not work.

@zubairahmed-ai
Copy link
Author

@Wanggcong Makes sense but I dont have test data :(

@zubairahmed-ai
Copy link
Author

@Wanggcong I downloaded net_last.pth and pytorch_result2.mat files and using them in evaluate_st.py but I am getting the error which is what I opened this issue with, can you please help?

@Wanggcong
Copy link
Owner

Wanggcong commented Jan 8, 2020

Without some important information, I afraid it is difficult for me to give you a good suggestion. You may try python -mpdb xxx.py for debugging the code and find the key problem. But this could be out of range of my code.

@zubairahmed-ai
Copy link
Author

I didnt change any of your code, I just grab your weights and run your file then got the above error, what do you want me to try?

@Wanggcong
Copy link
Owner

Do you have a GPU in your computer? How much is the memory? I confirm my code many times, there is no error like "out of memory" if setting batch size to 1

@zubairahmed-ai
Copy link
Author

zubairahmed-ai commented Jan 8, 2020

@Wanggcong I have a 1060 and I am not extracting features, I am using evaluate_st.py which loads your .mat files so no need to extract features (this) before that am I right?

@Wanggcong
Copy link
Owner

For anyone running this on Windows replace line 147 in test_st_market.py with this filename = path.split('\\')[-1] and line 153 with this if label[0:2]=='-1' or label[0:2]=='-1_c': and save yourself some headache/time

How about the step 3? You could have to rename the dataset when preparing the dataset, as described in readme.md.

'''
data prepare

change the path of dataset
python3 prepare.py --Market
'''

@zubairahmed-ai
Copy link
Author

I skipped step 3 because even with a batch size of 1 I am getting out of memory error

@Wanggcong
Copy link
Owner

@Wanggcong I have a 1060 and I am not extracting features, I am using evaluate_st.py which loads your .mat files so no need to extract features (this) before that am I right?

No, you have to extract features. '.mat' file is a st model and '.pth' is net weight.

@zubairahmed-ai
Copy link
Author

ok can you please share the extracted features then and show me how to use it? because acquiring better hardware is out of the question for me right now so all I got is 16Gb RAM and 1060

@Wanggcong
Copy link
Owner

No problem. Extracted features will upload.

@zubairahmed-ai
Copy link
Author

Thank you and please show how to use them to match images, I have to use them in a video

@Wanggcong
Copy link
Owner

Wanggcong commented Jan 8, 2020

Features will be uploaded into https://pan.baidu.com/s/1QMp22dVGJvBH45e4XPdeKw
I has mentioned if you use it to another dataset, you have to re-train the model. I suggest you read my paper and know the limitation of our method. If you use for video, that could need related data processing and other code. I am sorry that I cannot provide further code for that purpose.

@zubairahmed-ai
Copy link
Author

Ok that makes sense, the code only works on your own dataset. Thanks for explaining that, I would still try your model once ready on my own data

@khodabakhshih
Copy link

Dear Guangcong

Thanks for sharing your valuables implementation of your paper.

I read your “readme” files and FAQ in GitHub. I am new in python and it is rather wage for me.

I just want to test your code for some query images of market-1501. I have downloaded this data set before. Would you please tell me what exactly I must to do?

Thank you very much and looking forward to hearing you.

Regards

Hossein

@Wanggcong
Copy link
Owner

Wanggcong commented Jan 19, 2020

@khodabakhshih
Hi, Hossein

A. If you do not want to re-train a model, you can use our trained models, please read the discussion above. I summarize it as follows.

One key point is changing the paths, as shown in bold below.

--data prepare
change the path of dataset
python3 prepare.py --Market
--create a folder (suppose you name it A)
--pull the two pretrained models into that folder. (i.e., pytorch_result2.mat for the st model and net_last.pth for net weights) . baiduyun Link:https://pan.baidu.com/s/1QMp22dVGJvBH45e4XPdeKw password:dn7b
--extract features, test (appearance feature extraction)
python3 test_st_market.py --PCB --gpu_ids 2 --name A --test_dir "/home/huangpg/st-reid/dataset/market_rename/" --batchsize 8
note: **"/home/huangpg/st-reid/dataset/market_rename/" ** is your data path.

or use my extracted features https://pan.baidu.com/s/1QMp22dVGJvBH45e4XPdeKw.

--evaluate (joint metric, you can use your own visual feature or spatial-temporal streams)
python3 evaluate_st.py --name A

B. If you want to re-train the model, I think you just follow the third point (3. experiment) in my readme files (copied in the following), and do it step by step. One key point is changing the paths, shown in bold below.

--data prepare

change the path of dataset
python3 prepare.py --Market
--train (appearance feature learning)
python3 train_market.py --PCB --gpu_ids 2 --name ft_ResNet50_pcb_market_e --erasing_p 0.5 --train_all --data_dir "/home/huangpg/st-reid/dataset/market_rename/"

--test (appearance feature extraction)
python3 test_st_market.py --PCB --gpu_ids 2 --name ft_ResNet50_pcb_market_e --test_dir "/home/huangpg/st-reid/dataset/market_rename/" --batchsize 8

--generate st model (spatial-temporal distribution)
python3 gen_st_model_market.py --name ft_ResNet50_pcb_market_e --data_dir "/home/huangpg/st-reid/dataset/market_rename/"

--evaluate (joint metric, you can use your own visual feature or spatial-temporal streams)
python3 evaluate_st.py --name ft_ResNet50_pcb_market_e

--re-rank
6.1) python3 gen_rerank_all_scores_mat.py --name ft_ResNet50_pcb_market_e
6.2) python3 evaluate_rerank_market.py --name ft_ResNet50_pcb_market_e


Best wishes,

Guangcong

@khodabakhshih
Copy link

@Wanggcong Thank you in advance.

@khodabakhshih
Copy link

Hi
Have a good time. In the real world re-id applications how can we know if the query bbox appears in the gallery? The scores are so close, do you have any idea?
Regards

@DGxiansheng
Copy link

@Wanggcong hi,can i use this model without gpu? if it is possible,what should i do?

@DGxiansheng
Copy link

你好,你训练好的这个模型是可以直接用吧?我看了别人提的问题和你的回答,现在有个需求,就是想用这个模型识别视频里面的特定的人,看了会儿,有点懵,可以说说怎么搞吗?我英文贼菜。

@Wanggcong
Copy link
Owner

你好,你训练好的这个模型是可以直接用吧?我看了别人提的问题和你的回答,现在有个需求,就是想用这个模型识别视频里面的特定的人,看了会儿,有点懵,可以说说怎么搞吗?我英文贼菜。

--模型针对对应的数据集,如果要换数据集,需要重新训练,包括视觉特征和时空分布统计。

@Jasper0191
Copy link

Hello Wanggcong
You have done a excellent and impressive work. Actually I am new in machine learning and I was trying to run the code but I was facing problems. It would be grateful if you can suggest any YouTube link or upload video on how to run after downloading the code and dataset.

@umutpiri
Copy link

umutpiri commented Dec 17, 2020

I skipped step 3 because even with a batch size of 1 I am getting out of memory error

You can try to decrease the number of dataloader workers at test_st_duke.py.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants