Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added Observability features #64

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

JotaDeRodriguez
Copy link

I propose adding these economic metrics to the negotiation environment so that we could study the turn by turn performance of both players as the game develops. This could be particularly important to provide a baseline and evaluate specialized models down the line.

Some of the key metrics are:

  • The value for each player both in the offer and requests
  • The amount of resources offered / requested
  • whether or not the trades are valuable for one, both or neither player
  • gains / losses for both players
  • the value creation / value efficiency of the trade
  • overall results of the game
  • etc

Ideally a smarter / more capable model should be able to extract more value from less capable models. We could even set the models to find mutually beneficial solutions and use these metrics to ensure that the overall value for both players goes up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant