dealing with mjWARN_BADQACC

[vittorione94]

Hello,

This is more a feature enhancement. For environments that are unstable i.e. that rarely return a mjWARN_BADQACC would be great if instead of abruptly stopping the training would return a termination state.

I'm thinking wrapping this line in a try catch

agents/tf_agents/drivers/dynamic_step_driver.py

Line 137 in a8bd037

next_time_step = self.env.step(action_step.action)

What do you think is it something that would make sense?

Best,
-Vittorio

[vittorione94]

At the end I'm using this fix:

    try:
      ts = convert_time_step(self._env.step(action))
    except:
      observation = self._env._task.get_observation(self.physics)
      reward = 0.
      discount = 0.9
      ts = dm_env.TimeStep(
          dm_env.StepType.LAST, reward, discount, observation)
      ts = convert_time_step(ts)
    return ts

inside dm_control_wrapper, this is the line I modified:

agents/tf_agents/environments/dm_control_wrapper.py

Line 91 in 6d0ff50

return convert_time_step(self._env.step(action))