Fix wrong returns in multi agent partially returned episodes

[drblallo]

Why are these changes needed?

There is a bug in the way returns are calculated when a episode has partially returned chucks. In that situation instead of using the previously cached returns of the current agent, it uses the sum of rewards of all agents.

Related issue number

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it in doc/source/tune/api/ under the
    corresponding .rst file.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[drblallo]

has there been any progress of this? It is a very simple fix, and the bug deeply breaks the multi agent setup.

[drblallo]

@sven1977 can you take a look at this bug report? The issue has been introduced by a recent commit of yours 611e7d5 and taking care of it should be a very easy fix.

What is happening is just that the returns of all agents are mixed together, instead of just using the returns of the current agent

[drblallo]

addressed by #45543 , tnx!