ValueError: Only scalar actions are supported now!!

[shamim237]

I am getting this error while trying to create a custom environment and feed it to the DQN agent.

ValueError: Only scalar actions are supported now, but action spec is: BoundedTensorSpec(shape=(1,), dtype=tf.float32, name='action', minimum=array(0., dtype=float32), maximum=array(1., dtype=float32))
In call to configurable 'DqnAgent' (<class 'tf_agents.agents.dqn.dqn_agent.DqnAgent'>)

Here is my action_spec:

self._action_spec = array_spec.BoundedArraySpec(shape = (1,), dtype = np.float32, minimum = 0, maximum = 1, name = 'action')

[sguada]

Change your action spec to
array_spec.BoundedArraySpec(shape = (), dtype = np.float32, minimum = 0, maximum = 1, name = 'action')

[ayarmak]

I'm getting a similar error, but I cannot apply the solution by @sguada above, because I need my action to be a tuple of two integers.

My action_spec:

self._action_spec = array_spec.BoundedArraySpec(
        shape=(2,),
        dtype=np.int32, minimum=0, maximum=4, name='action')

The error I'm getting:

ValueError: Only scalar actions are supported now, but action spec is: BoundedTensorSpec(shape=(2,), dtype=tf.int32, name='action', minimum=array(0, dtype=int32), maximum=array(4, dtype=int32))
  In call to configurable 'DqnAgent' (<class 'tf_agents.agents.dqn.dqn_agent.DqnAgent'>)

Any advice on what I might be doing wrong?

[sguada]

As the error message shows, only scalar actions are supported for DQN, you can however transform your environment to have 1 action which represents the cross-product of all your actions.

[sguada]

Yeah you could manually convert the tuple of actions back and forth, or use FlattenActionWrapper https://github.com/tensorflow/agents/blob/master/tf_agents/environments/wrappers.py#L274 To do it automatically. Sergio

…

On Mon, Oct 11, 2021 at 12:19 PM Andrei Yarmak ***@***.***> wrote: Or do you mean something like action = action1 * 10 + action2, and then action1, action2 = action // 10, action % 10? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#665 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AANPI7E2HH7UZ3RX5TL4HGTUGM2CXANCNFSM5FB5MRBQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[ayarmak]

Thanks a lot - this is super helpful @sguada !!

[jacklu333333]

Hi, @sguada
I have the same error, but I am still a little bit confused. I try the automatic method with FlattenActionWrapper. My understanding is that FlattenActionWrapper will automatically flatten the action, and then I have to convert it to TFPyEnvironment so that array_spec will become tensor_spec.

However, when I tried the following code

train_py_env = myenv()

train_env = tf_py_environment.TFPyEnvironment(
    wrappers.FlattenActionWrapper(
        train_py_env,
        np.uint8
    ), 
    isolation=True
)  

the following error pop up

only size-1 arrays can be converted to Python scalars
It seems the FlattenActionWrapper is ignored by TFPyEnvironment somehow. I couldn't find any other example online. I hope you can point out where did I do wrong.

Thanks in advance.

Best Regards,
Jack Lu

[KumarRupesh]

Hi, my action need to be a tuple of three integers. I am using a DQN agent and a custom PyEnvironment environment. I am also getting same error as Jack has shown above when I use FlattenActionWrapper. Please let me know if some one was able to resolve this.