nan value

[Gavinic]

when i try to train a thai model, there are some nan/inf accrued in DeformableTransformer.

and i check the out put of self.detection_transformer:
the output is [nan, nan, nan,......]
so i go to deeper:
i have found that the problem is the forward of 'DeformableTransformer', but i don know how to fix it.

my config file is blew:
BASE: "../Base_det.yaml"

MODEL:
WEIGHTS: ""
TRANSFORMER:
VOC_SIZE: 230
NUM_POINTS: 50

DATASETS:
TRAIN: ("thaicard_train",)
TEST: ("thaicard_eval",)

SOLVER:
IMS_PER_BATCH: 4
BASE_LR: 1e-4
LR_BACKBONE: 1e-4
WARMUP_ITERS: 0
STEPS: (320000,)
MAX_ITER: 375000
CHECKPOINT_PERIOD: 100000

TEST:
EVAL_PERIOD: 10000

OUTPUT_DIR: "output/vitaev2_s/thai_pre"

How to fix it ? Thank you!

[Gavinic]

here is my train.json file.


thank you!

[xiaoming-master]

Modify configuration file parameters：MODEL.TRANSFORMER.VOC_SIZE: [yourself voc size]