D4PG: Missing specification of critic variables in update operation?

[DevMMI]

Hey, thanks for this amazing repository! I was wondering where exactly the critic variables are being specified when building the critic update operation, and whether this was a possible accidental omission.

` # Take the mean loss on the batch
critic_loss = tf.negative(critic_loss)
critic_loss = tf.reduce_mean(critic_loss)
critic_loss += l2_regularization(self.critic_vars)

    # Gradient descent
    critic_trainer = tf.train.AdamOptimizer(Settings.CRITIC_LEARNING_RATE)

self.critic_train_op = critic_trainer.minimize(critic_loss)`

shouldn't it be

self.critic_train_op = critic_trainer.minimize(critic_loss, var_list=self.critic_vars)

in order to apply the loss function to minimize the variables? Thank you and please let me know if I'm incorrect.

[Valentin-Guillet]

Hey, thanks for your interest !
The argument var_list=self.critic_vars is not necessary as by default, tensorflow uses the variables in GraphKeys.TRAINABLE_VARIABLES to update (see here).
Here, the trainable variables are the critic's and the actor's ones but the gradient of the critic loss on the actor variables is null, so it is not necessary to remove them.
I hope that this is clear, and feel free to ask any other question !