Problems of calculating FGD

[Jacoo-ai]

Hi, it's a really good work!

But I am confused about the FGD value. When I change the "val_data_path" to 'data/ted_dataset/lmdb_test' ,
and run "python scripts/synthesize.py eval output/train_multimodal_context/multimodal_context_checkpoint_best.bin" for several times, the result is
"2024-03-17 00:47:48,255: [VAL] loss: 0.075, joint mae: 0.02742, accel diff: 0.00322, FGD: 0.0000011660, feat_D: 0.010 / 25.9s". The FGD value is really small, only 0.0000011660, which is not similar with 3.729.

Is this the correct way to calculate the FGD value?

[youngwoo-yoon]

Hello,
Your modification seems correct, but the test set is supposed to give similar results with the validation set. I remember I got similar results.