BinoHeM: Binocular Singular Hellinger Metametric for Fine-Grained Few-Shot Classification

This package includes our codes for implementing "BinoHeM: Binocular Singular Hellinger Metametric for Fine-Grained Few-Shot Classification". (First Release Date: 2024-5-28)

1. Introduction

Meta-metric methods benefit from the advantages of concise principles but high efficiency, yet we also found that current meta-metric methods perform not ideal with the fine-grained scenarios. Inspired by the specialty of our human binocular visual systems, this paper originally proposes a human-like metametric paradigm, the Binocular singular Hellinger Metametric (BinoHeM), with the superior symmetric binocular encoding and recognition mechanisms. Specificaly, we have innovatively put forward two Binocular Sensing Feature Encoders(BSFEs), Singular Hellinger Metametric (HeM), and two Collaborative Identification Mechanisms(CIMs). On this basis, we further advance two excellent variants(BinoHeM-KDL,BinoHeM-MTL), adopting two different training mechanisms: knowledge distillation learning and meta-transfer learning.

2. Pre-Training, Meta/KDL-Training, Meta/KDL-Evaluation for Our BinoHeM

##############################################
# 1.Pretraining for BinoHeM-MTL & BinoHeM-KDL.
##############################################
gpuid=3
dataset='stanford_car'
model='Vision_EResnet12'
method='BinoHeD'
vision_method='disparity' #['pure','disparity']
FLLS='shared'
AELS='shared'
DRS='SVD'     #['GCM','GRM','SVD']
CIM='WAS'     #['WAS', 'ARS']
check_resume='False'
check_epoch=0
# 预训练超参：
optim='SGD'   #['SGD','Adam']
pre_batch_size=32
pre_lr=0.05
pre_epoch=170
pre_num_episode=50
if [ $model != 'ResNet12' ]; then
    model_l='Le_'$model
    model_r='Re_'$model
else
    model_l=$model
    model_r=$model
fi
DATA_ROOT=/home/ssdData/qcfData/BinoHeD_benchmark/$dataset
cd ../

echo "============= pre-train ============="
python bino1_pretrain.py --CIM $CIM --FLLS $FLLS --AELS $AELS --DRS $DRS --batch_size $pre_batch_size --dataset $dataset --data_path $DATA_ROOT --model_l $model_l --model_r $model_r --method $method \
--vision_method $vision_method --check_resume $check_resume --check_epoch $check_epoch --optimizer $optim --image_size 84 --gpu ${gpuid} --pre_lr $pre_lr --wd 1e-4 \
--epoch $pre_epoch --milestones 100 150 --save_freq 100 --val meta --val_n_episode $pre_num_episode --n_shot 5

###############################################
# 2.Meta-training on 5way-1shot for BinoHeM-MTL
###############################################
gpuid=3
dataset='stanford_car'
model='Vision_EResnet12'
method='BinoHeD' 
vision_method='disparity'
FLLS='shared' 
AELS='shared' 
DRS='SVD'     #['GCM','GRM','SVD']
CIM='WAS'     #['WAS', 'ARS']
check_resume='False'
check_epoch=0
# 预训练超参：
pre_optim='SGD'   #['SGD','Adam']
pre_batch_size=32
pre_lr=0.05
pre_epoch=170
pre_num_episode=50
# 元训练超参：
optim='SGD'   #['SGD','Adam']
meta_lr=0.0001
meta_epoch=80
meta_train_episode=1000
meta_val_episode=600
if [ $model != 'ResNet12' ]; then
    model_l='Le_'$model
    model_r='Re_'$model
else
    model_l=$model
    model_r=$model
fi
if [ $FLLS = 'shared' ]; then
    FLLS_='share'
else
    FLLS_='split'
fi
if [ $AELS = 'shared' ]; then
    AELS_='share'
else
    AELS_='split'
fi
DATA_ROOT=/home/ssdData/qcfData/BinoHeD_benchmark/$dataset
MODEL_PATH=./checkpoints/$dataset/Bino_${model}_${method}_${vision_method}/FLLS_${FLLS_}_AELS_${AELS_}_DRS_${DRS}_CIM_${CIM}/prephase_${pre_optim}_${pre_batch_size}_${pre_lr}_${pre_num_episode}_${pre_epoch}/best_model.tar
cd ../

echo "============= meta-train 1-shot ============="
python bino2_metatrain.py --pre_batch_size $pre_batch_size --pre_epoch $pre_epoch --pre_num_episode $pre_num_episode --CIM $CIM --FLLS $FLLS --AELS $AELS --DRS $DRS --dataset $dataset --data_path $DATA_ROOT --model_l $model_l --model_r $model_r --method $method \
--pre_optimizer $pre_optim --vision_method $vision_method --check_resume $check_resume --check_epoch $check_epoch --optimizer $optim --image_size 84 --gpu ${gpuid} --pre_lr $pre_lr --meta_lr $meta_lr \
--epoch $meta_epoch --milestones 40 80 --n_shot 1 --train_n_episode $meta_train_episode --val_n_episode $meta_val_episode --pretrain_path $MODEL_PATH

###############################################
# 2.KDL-training on 5way-1shot for BinoHeM-KDL
###############################################
gpuid=3
dataset='stanford_car'
model='Vision_EResnet12'
method='BinoHeD'
vision_method='disparity'
FLLS='shared'
AELS='shared'
DRS='SVD'     #['GCM','GRM','SVD']
CIM='WAS'     #['WAS', 'ARS']
check_resume='False'
check_epoch=0
# 预训练超参：
pre_optim='SGD'  
pre_batch_size=32
pre_lr=0.05
pre_epoch=170
pre_num_episode=50
# 蒸馏超参：
kdl_optim='Adam'
kdl_epoch=170

if [ $model != 'ResNet12' ]; then
    model_l='Le_'$model
    model_r='Re_'$model
else
    model_l=$model
    model_r=$model
fi
if [ $FLLS = 'shared' ]; then
    FLLS_='share'
else
    FLLS_='split'
fi
if [ $AELS = 'shared' ]; then
    AELS_='share'
else
    AELS_='split'
fi
DATA_ROOT=/home/ssdData/qcfData/BinoHeD_benchmark/$dataset
cd ../

####################
# 知识蒸馏1
####################
echo "============= 5Way-1Shot ============="
MODEL_PATH=./checkpoints/$dataset/Bino_${model}_${method}_${vision_method}/FLLS_${FLLS_}_AELS_${AELS_}_DRS_${DRS}_CIM_${CIM}/prephase_${pre_optim}_${pre_batch_size}_${pre_lr}_${pre_num_episode}_${pre_epoch}/best_model.tar
python bino4_distilltrain.py --CIM $CIM --FLLS $FLLS --AELS $AELS --DRS $DRS --batch_size $pre_batch_size --dataset $dataset --data_path $DATA_ROOT --model_l $model_l --model_r $model_r --method $method \
--vision_method $vision_method --check_resume $check_resume --check_epoch $check_epoch --optimizer $kdl_optim --image_size 84 --gpu ${gpuid} --pre_lr $pre_lr --wd 1e-4 \
--epoch $kdl_epoch --milestones 100 150 --save_freq 100  --val_n_episode $pre_num_episode --n_shot 1  --val meta  --teacher_path $MODEL_PATH --trial 1 --val last \
--distill_lr 0.05
####################
# 知识蒸馏2
####################
echo "============= 5Way-1Shot ============="
teacher_path1=./checkpoints/$dataset/Bino_${model}_${method}_${vision_method}/FLLS_${FLLS_}_AELS_${AELS_}_DRS_${DRS}_CIM_${CIM}/distill_born1_${kdl_optim}_${pre_batch_size}_${pre_lr}_${pre_num_episode}_${kdl_epoch}_5way_1shot/distill_train/last_model.tar
python bino4_distilltrain.py --CIM $CIM --FLLS $FLLS --AELS $AELS --DRS $DRS --batch_size $pre_batch_size --dataset $dataset --data_path $DATA_ROOT --model_l $model_l --model_r $model_r --method $method \
--vision_method $vision_method --check_resume $check_resume --check_epoch $check_epoch --optimizer $kdl_optim --image_size 84 --gpu ${gpuid} --pre_lr $pre_lr --wd 1e-4 \
--epoch $kdl_epoch --milestones 100 150 --save_freq 100  --val_n_episode $pre_num_episode --n_shot 1  --val meta  --teacher_path $teacher_path1 --trial 2 --val last \
--distill_lr 0.05

#################################################
# 3.Meta-evaluation on 5way-1shot for BinoHeM-MTL
#################################################
gpuid=3
dataset='stanford_car'
model='Vision_EResnet12'
method='BinoHeD' 
vision_method='disparity'
FLLS='shared'
AELS='shared'
DRS='SVD' #['GCM','GRM','SVD']
CIM='WAS' #['WAS', 'ARS']
# 预训练超参：
pre_optim='SGD'
pre_batch_size=32
pre_lr=0.05
pre_epoch=170
pre_num_episode=50
# 元训练超参：
optim='SGD'
meta_lr=0.0001
meta_epoch=80
meta_train_episode=1000
meta_val_episode=600
if [ $model != 'ResNet12' ]; then
    model_l='Le_'$model
    model_r='Re_'$model
else
    model_l=$model
    model_r=$model
fi
if [ $FLLS = 'shared' ]; then
    FLLS_='share'
else
    FLLS_='split'
fi
if [ $AELS = 'shared' ]; then
    AELS_='share'
else
    AELS_='split'
fi
DATA_ROOT=/home/ssdData/qcfData/BinoHeD_benchmark/$dataset
MODEL_1SHOT_PATH=./checkpoints/$dataset/Bino_${model}_${method}_${vision_method}/FLLS_${FLLS_}_AELS_${AELS_}_DRS_${DRS}_CIM_${CIM}/metaphase_${pre_optim}_${pre_batch_size}_${pre_lr}_${pre_num_episode}_${pre_epoch}_5way_1shot/metatrain_${optim}_${meta_lr}_${meta_epoch}_train_${meta_train_episode}_val_${meta_val_episode}/best_model.tar
cd ../

echo "============= meta-test 1-shot ============="
N_SHOT=1
python bino3_metatest.py --pre_epoch $pre_epoch --pre_num_episode $pre_num_episode --meta_train_n_episode $meta_train_episode --meta_val_n_episode $meta_val_episode --meta_epoch $meta_epoch --pre_batch_size $pre_batch_size --CIM $CIM --FLLS $FLLS --AELS $AELS --DRS $DRS --pre_lr $pre_lr --meta_lr $meta_lr --dataset $dataset --data_path $DATA_ROOT --model_l $model_l --model_r $model_r --method $method \
--pre_optimizer $pre_optim --vision_method $vision_method --optimizer $optim --image_size 84 --gpu ${gpuid} --n_shot $N_SHOT --reduce_dim 640 --model_path $MODEL_1SHOT_PATH --test_task_nums 3 --test_n_episode 1000

#################################################
# 3.KDL-evaluation on 5way-1shot for BinoHeM-KDL
#################################################
gpuid=3
dataset='stanford_car'
model='Vision_EResnet12'
method='BinoHeD'
vision_method='disparity'
FLLS='shared'
AELS='shared'
DRS='SVD'    #['GCM','GRM','SVD']
CIM='WAS'    #['WAS', 'ARS']
# 预训练超参：
pre_optim='SGD'
pre_batch_size=32
pre_lr=0.05
pre_epoch=170
pre_num_episode=50
# 蒸馏超参：
kdl_optim='Adam'
kdl_epoch=170
times_test=3
num_episode=1000
if [ $model != 'ResNet12' ]; then
    model_l='Le_'$model
    model_r='Re_'$model
else
    model_l=$model
    model_r=$model
fi
if [ $FLLS = 'shared' ]; then
    FLLS_='share'
else
    FLLS_='split'
fi
if [ $AELS = 'shared' ]; then
    AELS_='share'
else
    AELS_='split'
fi
DATA_ROOT=/home/ssdData/qcfData/BinoHeD_benchmark/$dataset
MODEL11_PATH=./checkpoints/$dataset/Bino_${model}_${method}_${vision_method}/FLLS_${FLLS_}_AELS_${AELS_}_DRS_${DRS}_CIM_${CIM}/distill_born1_${kdl_optim}_${pre_batch_size}_${pre_lr}_${pre_num_episode}_${kdl_epoch}_5way_1shot/distill_train/last_model.tar
MODEL21_PATH=./checkpoints/$dataset/Bino_${model}_${method}_${vision_method}/FLLS_${FLLS_}_AELS_${AELS_}_DRS_${DRS}_CIM_${CIM}/distill_born2_${kdl_optim}_${pre_batch_size}_${pre_lr}_${pre_num_episode}_${kdl_epoch}_5way_1shot/distill_train/last_model.tar
cd ../

####################
# 知识蒸馏1
####################
trial=1
if [ "$trial" -eq 1 ]; then
    MODEL1_PATH=$MODEL11_PATH
    MODEL5_PATH=$MODEL15_PATH
elif [ "$trial" -eq 2 ]; then
    MODEL1_PATH=$MODEL21_PATH
    MODEL5_PATH=$MODEL25_PATH
else
    echo "Please set the trial!"
fi
echo "============= meta-test 1-shot ============="
N_SHOT=1
python bino5_distilltest.py --pre_epoch $pre_epoch --pre_num_episode $pre_num_episode --pre_batch_size $pre_batch_size --CIM $CIM --FLLS $FLLS --AELS $AELS --DRS $DRS --pre_lr $pre_lr --dataset $dataset --data_path $DATA_ROOT --model_l $model_l --model_r $model_r --method $method \
--pre_optimizer $pre_optim --vision_method $vision_method --optimizer $kdl_optim --image_size 84 --gpu ${gpuid} --n_shot $N_SHOT --reduce_dim 640 --model_path $MODEL1_PATH --trial $trial --test_task_nums $times_test --test_n_episode $num_episode

####################
# 知识蒸馏2
####################
trial=2
if [ "$trial" -eq 1 ]; then
    MODEL1_PATH=$MODEL11_PATH
    MODEL5_PATH=$MODEL15_PATH
elif [ "$trial" -eq 2 ]; then
    MODEL1_PATH=$MODEL21_PATH
    MODEL5_PATH=$MODEL25_PATH
else
    echo "Please set the trial!"
fi
echo "============= meta-test 1-shot ============="
N_SHOT=1
python bino5_distilltest.py --pre_epoch $pre_epoch --pre_num_episode $pre_num_episode --pre_batch_size $pre_batch_size --CIM $CIM --FLLS $FLLS --AELS $AELS --DRS $DRS --pre_lr $pre_lr --dataset $dataset --data_path $DATA_ROOT --model_l $model_l --model_r $model_r --method $method \
--pre_optimizer $pre_optim --vision_method $vision_method --optimizer $kdl_optim --image_size 84 --gpu ${gpuid} --n_shot $N_SHOT --reduce_dim 640 --model_path $MODEL1_PATH --trial $trial --test_task_nums $times_test --test_n_episode $num_episode

3. Exp1: Typical Fine-Grained Few-shot Classification(5way-1shot, -5shot)

We report the Experimental results (Vision-EResnet12) on CUB-200-2011, Aircraft-fewshot, VGG-Flowers and StanfordCars benchmarks. We conducted three times 1,000 randomly sampled episodes experiments, and report average results for both 1-shot and 5-shot evaluation. More details on the experiments can be seen in the paper.

Ⅰ. CUB-200-2011

Method(Ours)	Vision-EResnet12		Pre-trained Models		Meta/KDL-trained Models		Model Evaluation Notes
Method(Ours)	5-way-1-shot	5-way-5-shot	Case-Model	Lab-Note-Taking	Case-Model(1shot)	Lab-Note-Taking	5-way-1-shot	5-way-5-shot
BinoHeM-KDL(ARS)	85.49±0.56	94.46±0.27	Download	Download	Download	Download	Download	Download
BinoHeM-KDL(WAS)	85.17±0.56	94.31±0.29	Download	Download	Download	Download	Download	Download
BinoHeM-MTL(ARS)	86.71±0.55	94.45±0.28	Download	Download	Download	Download	Download	Download
BinoHeM-MTL(WAS)	87.00±0.54	94.52±0.27	Download	Download	Download	Download	Download	Download

Note that for BinoHeM-MTL and BinoHeM-KDL, we used to obtain the same pre-trained models; and make experiments based on two CIMs for better results.

Ⅱ. Aircraft-fewshot

Method(Ours)	Vision-EResnet12		Pre-trained Models		Meta/KDL-trained Models		Model Evaluation Notes
Method(Ours)	5-way-1-shot	5-way-5-shot	Case-Model	Lab-Note-Taking	Case-Model(1shot)	Lab-Note-Taking	5-way-1-shot	5-way-5-shot
BinoHeM-KDL(ARS)	89.98±0.46	95.32±0.20	Download	Download	Download	Download	Download	Download
BinoHeM-KDL(WAS)	90.10±0.48	95.29±0.20	Download	Download	Download	Download	Download	Download
BinoHeM-MTL(ARS)	90.57±0.48	95.25±0.20	Download	Download	Download	Download	Download	Download
BinoHeM-MTL(WAS)	90.68±0.47	95.36±0.20	Download	Download	Download	Download	Download	Download

Note that for BinoHeM-MTL and BinoHeM-KDL, we used to obtain the same pre-trained models; and make experiments based on two CIMs for better results.

Ⅲ. VGG-Flowers

Method(Ours)	Vision-EResnet12		Pre-trained Models		Meta/KDL-trained Models		Model Evaluation Notes
Method(Ours)	5-way-1-shot	5-way-5-shot	Case-Model	Lab-Note-Taking	Case-Model(1shot)	Lab-Note-Taking	5-way-1-shot	5-way-5-shot
BinoHeM-KDL(ARS)	81.82±0.56	93.28±0.28	Download	Download	Download	Download	Download	Download
BinoHeM-KDL(WAS)	81.73±0.57	93.19±0.28	Download	Download	Download	Download	Download	Download
BinoHeM-MTL(ARS)	82.04±0.59	93.87±0.26	Download	Download	Download	Download	Download	Download
BinoHeM-MTL(WAS)	82.31±0.57	93.88±0.26	Download	Download	Download	Download	Download	Download

Note that for BinoHeM-MTL and BinoHeM-KDL, we used to obtain the same pre-trained models; and make experiments based on two CIMs for better results.

Ⅳ. Stanford-Cars

Method(Ours)	Vision-EResnet12		Pre-trained Models		Meta/KDL-trained Models		Model Evaluation Notes
Method(Ours)	5-way-1-shot	5-way-5-shot	Case-Model	Lab-Note-Taking	Case-Model(1shot)	Lab-Note-Taking	5-way-1-shot	5-way-5-shot
BinoHeM-KDL(ARS)	81.46±0.63	93.81±0.28	Download	Download	Download	Download	Download	Download
BinoHeM-KDL(WAS)	81.08±0.63	93.84±0.28	Download	Download	Download	Download	Download	Download
BinoHeM-MTL(ARS)	85.09±0.56	94.32±0.26	Download	Download	Download	Download	Download	Download
BinoHeM-MTL(WAS)	84.90±0.57	94.36±0.26	Download	Download	Download	Download	Download	Download

Note that for BinoHeM-MTL and BinoHeM-KDL, we used to obtain the same pre-trained models; and make experiments based on two CIMs for better results.

4. Exp2: Generalized Fine-Grained Few-shot Classification(multi-way,multi-shot)

We report the Experimental results on StanfordCars benchmark. We conducted three times 1,000 randomly sampled episodes experiments, and report average results for both Multi-way(5-10) and Multi-shot(1,5,10,15,20,25) evaluation. More details on the experiments can be seen in the paper.

Method(Ours)	5way-1shot			6way-1shot			7way-1shot			8way-1shot
Method(Ours)	ACC.(%)	Pre-train	Meta-train	ACC.(%)	Pre-train	Meta-train	ACC.(%)	Pre-train	Meta-train	ACC.(%)	Pre-train	Meta-train
Protonet	78.25 [Note]	Model [Note]	Model [Note]	75.55 [Note]	Model [Note]	Model [Note]	73.19 [Note]	Model [Note]	Model [Note]	70.78 [Note]	Model [Note]	Model [Note]
DeepBDC	76.84 [Note]	Model [Note]	Model [Note]	74.61 [Note]	Model [Note]	Model [Note]	72.44 [Note]	Model [Note]	Model [Note]	70.68 [Note]	Model [Note]	Model [Note]
BinoHeM(Ours)	84.90 [Note]	Model [Note]	Model [Note]	82.77 [Note]	Model [Note]	Model [Note]	80.90 [Note]	Model [Note]	Model [Note]	79.19 [Note]	Model [Note]	Model [Note]

Method(Ours)	5way-5shot			5way-10shot			5way-15shot			5way-20shot
Method(Ours)	ACC.(%)	Pre-train	Meta-train	ACC.(%)	Pre-train	Meta-train	ACC.(%)	Pre-train	Meta-train	ACC.(%)	Pre-train	Meta-train
Protonet	89.74 [Note]	Model [Note]	Model [Note]	91.61 [Note]	Model [Note]	Model [Note]	92.87 [Note]	Model [Note]	Model [Note]	93.48 [Note]	Model [Note]	Model [Note]
DeepBDC	89.63 [Note]	Model [Note]	Model [Note]	93.08 [Note]	Model [Note]	Model [Note]	93.90 [Note]	Model [Note]	Model [Note]	94.54 [Note]	Model [Note]	Model [Note]
BinoHeM(Ours)	94.32 [Note]	Model [Note]	Model [Note]	95.57 [Note]	Model [Note]	Model [Note]	96.01 [Note]	Model [Note]	Model [Note]	96.26 [Note]	Model [Note]	Model [Note]

5. License & disclaimer

The codes can be used for research purposes only. This package is strictly for non-commercial academic use only.

6. Acknowledgment

Our code refers the corresponding code publicly available: DeepBDC

Name		Name	Last commit message	Last commit date
Latest commit History 32 Commits
assets		assets
data		data
methods		methods
network		network
LICENSE.txt		LICENSE.txt
README.md		README.md
bino1_pretrain.py		bino1_pretrain.py
bino2_metatrain.py		bino2_metatrain.py
bino3_metatest.py		bino3_metatest.py
bino4_distilltrain.py		bino4_distilltrain.py
bino5_distilltest.py		bino5_distilltest.py
pip_apex.sh		pip_apex.sh
pip_reqs.sh		pip_reqs.sh
utils.py		utils.py

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

BinoHeM: Binocular Singular Hellinger Metametric for Fine-Grained Few-Shot Classification

1. Introduction

2. Pre-Training, Meta/KDL-Training, Meta/KDL-Evaluation for Our BinoHeM

3. Exp1: Typical Fine-Grained Few-shot Classification(5way-1shot, -5shot)

Ⅰ. CUB-200-2011

Ⅱ. Aircraft-fewshot

Ⅲ. VGG-Flowers

Ⅳ. Stanford-Cars

4. Exp2: Generalized Fine-Grained Few-shot Classification(multi-way,multi-shot)

5. License & disclaimer

6. Acknowledgment

About

Releases

Packages

Languages

License

ChaofeiQI/BinoHeM

Folders and files

Latest commit

History

Repository files navigation

BinoHeM: Binocular Singular Hellinger Metametric for Fine-Grained Few-Shot Classification

1. Introduction

2. Pre-Training, Meta/KDL-Training, Meta/KDL-Evaluation for Our BinoHeM

3. Exp1: Typical Fine-Grained Few-shot Classification(5way-1shot, -5shot)

Ⅰ. CUB-200-2011

Ⅱ. Aircraft-fewshot

Ⅲ. VGG-Flowers

Ⅳ. Stanford-Cars

4. Exp2: Generalized Fine-Grained Few-shot Classification(multi-way,multi-shot)

5. License & disclaimer

6. Acknowledgment

About

Resources

License

Stars

Watchers

Forks

Releases

Packages 0

Languages

Packages