Run several experiments using the default hyper-parameters but not getting the result in the paper.

[IDoMathEveryDay]

Hi, I replicated the mujoco experiments by setting '--natural = false' and all missing modal ratios to 0. All other hyper-parameters are as default as written in config.py. The network is trained from scratch without using pre-trained weights. This should be a baseline experiment using the MuMMI algorithm. But I get the following results, which are totally not aligned with the paper:
For the walker stand:

it seems alright.

For the walker walk:

The agent performance drops in the later training stage.

For the walker run:

The performance totally drops to the ground.

For finger spinning:

The agent basically learns nothing.

What am I doing wrong?
Could you tell me the right setting to replicate the result in your paper?

[kaiqi-ken]

Hi, we tested on --natural=false with missing observations and it should work well. Can you try on missing observations and see how it goes?

You can also try to set --natural=true and missing ratio same as the paper. Is it possible to replicate the results? I have uploaded files required to run --natural=true.

Do let me know if you have other questions.

[kaiqi-ken]

Besides, you can try reducing the weight of contrastive loss on touch modality.

[4078730]

In my environment, the network was successfully trained as in the paper under the condition "missing modal ratios = 0.05", but the network training was unstable under the condition "missing modal ratios = 0.00".
Do you have any idea why MuMMI does not perform best under the condition "missing modal ratios = 0"?

[kaiqi-ken]

In my environment, the network was successfully trained as in the paper under the condition "missing modal ratios = 0.05", but the network training was unstable under the condition "missing modal ratios = 0.00". Do you have any idea why MuMMI does not perform best under the condition "missing modal ratios = 0"?

[kaiqi-ken]

In my environment, the network was successfully trained as in the paper under the condition "missing modal ratios = 0.05", but the network training was unstable under the condition "missing modal ratios = 0.00". Do you have any idea why MuMMI does not perform best under the condition "missing modal ratios = 0"?

Hi, in our experiments, we find that 1) Contrastive learning requires more diverse data distribution. 2) Using MSE as a similarity metric is beneficial for learning a shared representation but sometimes it is not rich enough as a Cosine similarity and 3) The prediction model learned via contrastive loss is not good as reconstruction loss. The prior work (CVRL) combined model-free and model-based policy optimization to solve this problem.

[IDoMathEveryDay]

Interesting... Let me try "missing modal ratios = 0.05" and figure out what's happening.